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Abstract: In this paper, we begin a quantization program for nilpotent orbits of a real 
semisiniple Lie group G^. These orbits arise naturally as the coadjoint orbits of which are 
stable under scaling, and thus they have a canonical symplectic structure lo where the G]R-action is 
Hamiltonian. These orbits and their covers generalize the oscillator phase space T*M", which occurs 
here when = Sp{2n, M) and 0]r is minimal. 

A complex structure J polarizing Ok and invariant under a maximal compact subgroup of Gu 
is provided by the Kronheimer-Vergne Kaehler structure {3,iv). We argue that the Kaehler potential 
serves as the Hamiltonian. Using this setup, we realize the Lie algebra gjj of G^ as a Lie algebra of 
rational functions on the holomorphic cotangent bundle T*Y where Y = (0]r, J). 

Thus we transform the quantization problem on Ok into a quantization problem on T*Y. We 
explain this in detail and solve the new quantization problem on T*Y in a uniform manner for minimal 
nilpotent orbits in the non-Hermitian case. The Hilbert space of quantization consists of holomorphic 
half- forms on Y . We construct the reproducing kernel. The Lie algebra acts by explicit pseudo- 
differential operators on half-forms where the energy operator quantizing the Hamiltonian is inverted. 
The Lie algebra representation exponentiates to give a minimal unitary ladder representation of a 
cover of Or. Jordan algebras play a key role in the geometry and the quantization. 

§1. Introduction. 



I. Quantization of Phase Space. Quantization of a classical phase space M with 
symplectic form u; is a process whereby observables 4> are converted into self-adjoint operators 
Q{(j)) on a Hilbert space TC of states. The observables are simply the smooth functions on M. 

The Hilbert space H should arise, according to the philosophy of Geometric Quantization, 
as a space of polarized sections of a suitable complex line bundle over M. A real (complex) 
polarization of M consists of a integrable Lagrangian distribution inside the (complexified) 
tangent bundle. A polarized section, of a bundle with connection, is a section annihilated 
by all vector fields lying in the polarization; in the real case, this means that the section is 
covariantly constant along the leaves of the corresponding Lagrangian foliation. 

We require that the quantization satisfies Dirac's axioms (see e.g., [Ki], [A-M]) in some 
form. Dirac's consistency axiom is that the Poisson bracket of functions on M goes over into 
the commutator of operators so that 



Q({^,V.}) = i[Q(0),Q(^)] (LI) 
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(We have set ^ = 1.) Additional axioms mandate that the constant function 1 quantizes to 
the identity operator, and a complete set of observables quantizes to give a complete set of 
operators. 

In Hamiltonian mechanics, the physics of the system in encoded in a single observable F 
(usually written as E or H) called the Hamiltonian. Often F is the total energy. 

Any observable cj) generates a Hamiltonian flow: this is the flow of the Hamiltonian vector 
field deflned by the equation 

^^Ja; + d(/) = (1.2) 

The Poisson bracket on C°°{M) is given by {</>, V'} = C<)!>(V') = '^(C</>) Cv')- 

The Hamiltonian flow of the F gives the time evolution of the physical system. For any 
observable (j), the time derivative of as the system evolves is given by = {F, (/»}. This is 
a concise version of Hamilton's equations. On physical grounds, in certain circumstances, F 
should be a positive function on M. 

In quantization of a Hamiltonian mechanical system, F should be promoted to a self- 
adjoint operator Q{F) on 7i with positive spectrum. When F is the total classical energy, 
the spectrum of Q{F) should be discrete and give the possible quantized energy levels of the 
quantum system. 

II. Quantization of the n-dimensional Harmonic Oscillator. The most familiar 
model situation is the case where M is the cotangent bundle of some (conflguration) manifold 
X and uj is the canonical symplectic form so that u = AO where 6 is the Liouville 1-form 
on T*X. In this case we have the manifest cotangent polarization where the leaves are the 
cotangent spaces of X . We expect W to be a space of square integrable half-forms on X 
(see §2 and below starting around (1.5)). A smooth function f on X quantizes to a give a 
multiplication operator on H. If is a vector field on X, then the symbol quantizes to the 
Lie derivative Crj operator on half-forms. Consistent quantization of additional observables is 
problematic, as we see already in the oscillator example below. 

A second model situation is the case where M is a Kaehler manifold and u) is the Kaehler 
form. Then the complex structure J of M gives a complex polarization. Now "polarized" 
simply means "holomorphic" . Thus H should be a space of holomorphic square-integrable 
sections of a suitable holomorphic complex line bundle over M. 

The most familiar example of a Hamiltonian mechanical system, the oscillator phase space, 
admits both cotangent and Kaehler polarizations. The oscillator phase space is M = r*R". 
The canonical coordinates on T*R" are the position coordinates qi,... ,qn together with the 
momentum coordinates pi, . . . ,pn. The canonical symplectic form is w = Y2k=i '^Pk^dqk- The 
Poisson bracket satisfies {pj,Pk} = {qj,qk} = and {pj,qk} = ^jk- For general observables 
we have the classical formula 



n . 

-«=e( 

fc=l ^ 



dpk dqk dpk dqk 



In physics, T*W^ arises as the phase space of n uncoupled harmonic oscillators with Hamil- 
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tonian equal to the total energy (kinetic plus potential) 



F = 1^Y.^pI + qI) (1-4) 
fe=i 

We also have a natural Kaehler structure. We identify T*M"' = R^" = C" so that the 
complex- valued observables Zk = {pk +*'Zfc)/\/2 are holomorphic coordinates. Now is a 
Kaehler manifold with Kaehler form uj and Kaehler metric g = X]fe=i(dp^ + dq^). In the 
Zj^'Zk coordinates we have u) = iYlk=i '^^k A dzk and the Poisson bracket satisfies {zj, Zk} = 
{zj,Zk} = and {zj,Zk} = iSjk- Also (1.4) becomes 



fc=i 



The quantization of the Kaehler phase space M = C" gives the Fock-Bargmann model 
of the quantum mechanical oscillator. (Quantization by means of the real cotangent polar- 
ization gives the Schroedinger model.) In this model, TL is a space of holomorphic functions 
f{zi, . . . , Zn) on C". The Hamiltonian F quantizes into the energy operator 

The functions Zk and 'Zk quantize into the creation and annihilation operators 

d 

Q{zk) = Zk and Q(zfc) = — (1.7) 

OZk 

Then Q{F) is a grading operator on the quantum space and Q{zk) = Zk and Q(^fc) sue raising 
and lowering operators moving the eigenspaces of Q{F). 

One way to "explain" the |-shift in (1.6) (a quantum correction) is to adopt the sym- 
metrization procedure of canonical quantization so that 

Q{zkZk) = \ {Q{zk)Q{zk) + Q{zk)Q{zk)) = ^ (^fe^ + ^^'^) = ^-^^ + ^ ^^-^^ 

There is a unique Hermitian inner product {f\g) on the space H = C[zi, . . . ,Zn] of poly- 
nomial functions such that the operators Q{zk) and Q{zk) in (1.7) are mutually adjoint. (The 
condition that Q{4>) is self-adjoint for real (f) amounts to the condition that Q{(f)) and Q(^) are 
mutually adjoint for complex (p.) This inner product is positive definite with 

\\z'^'---z^f = a^[---aj. (1.9) 

The inner product (1.9) is given by the integral formula 

{f\g)= I /(z)^e-l^lVzdzl (1.10) 

JC^n 
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and this expression defines the inner product on the Hilbert space completion Ti of H. Thus 
Ti consists of all the holomorphic functions f{zi, . . . ,Zn) on which are "square integrable" 
in the sense that ||/||^ = (/|/) is finite. 

The reproducing kernel (see §8) of H is the holomorphic function ]C{z,w) on C" x C" 

)C{Z,W) = exp{ziWi H h ZnWn) (1-11) 

Here X denotes the complex conjugate manifold to a complex manifold X, so that holomorphic 
functions on X identify with anti-holomorphic functions on X. X is obtained from X by 
reversing the sign of the complex structure. 

The Hamiltonian flow of F lies inside a larger symmetry. The Hamiltonian F sits inside 
the space g of all homogeneous quadratic polynomials zi, . . . , Zn,^i, ■ ■ ■ ,Zn- The space g is a 
finite-dimensional Lie subalgebra of complex- valued observables under Poisson bracket. The 
Lie algebra g breaks naturally into 3 pieces: g = t © p"*" © p~ where 

t = span of ZjZk, p"*" = span of zjZk, p~ = span of zjZk (1-12) 

Here t arises as the subspace of G g which Poisson commute with F so that (f) is a conserved 
quantity. Then p+ and p~ are the irreducible ^-representations in g complementary to t. 

The subspace gK C g of real- valued observables is a Lie algebra real form of g. We have 
gM = © pM where 

tm=spsLnoi ZjZk + ZjZk and i{zjZk-ZjZk) 

(1.13) 

p^=s-psjioi ZjZk + ZjZk and i{zjZk-ZjZk) 

As Lie algebras, 6k ~ u(n), i ~ g((n, C), g^ ~ sp(2n,M) and g ~ sp(2n,C). 

The Hamiltonian flow of t^. on M = is the natural linear representation of the unitary 
group U{n). The Hamiltonian flow of g^ is the natural linear representation of the non- 
compact symplectic group Sp{2n, R). Clearly U{n) is exactly the subgroup of Sp{2n, M) which 
preserves the Hamiltonian F in (1.5). 

We can quantize all the observables in g, in a way consistent with (1.7) and Q(l) = 1, by 

Q{zjZk) = ZjZk, ^^^J'^fe) = + Q(^i^fc) = dzjdzk ^^'^^^ 

These operators obey (1.1) and Q{(t>)^ = Q{(t>) for ((> ^ Q- Moreover this condition by itself 
determines the inner product {f\g) uniquely. The No-Go Theorem (see e.g., [A-M]) shows that 
we cannot extend the quantization to all polynomial observables. 

A benefit of looking at this large Lie algebra of symmetry is that we can see another source 
for the i-shift in (1.6). Indeed, the eminently reasonable values of Q{zjZk) and Qi^j'Zk) in 
(1.14) imply the value of Q{zj'Zk) because of the Dirac axiom (1.1). So the term involving ^ 
is created exactly because ^ and Zk do not commute but instead [-§^,Zk\ = 1. 

The most convincing way to understand the ^-shift is to introduce half-forms. This means 
that we replace our Hilbert space Ti of holomorphic functions on C" by a new Hilbert space 
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H' of holomorphic half-forms s = f^Jv where / = f{z\, • • • , Zn) is still a holomorphic function 
and 

V = dzi f\ ■ ■ ■ f\ dzn (1-15) 

is a holomorphic n-form. Then every holomorphic vector field r] acts naturally on half-forms 

by the Lie derivative Crj (see §5). 

On half-forms, Zj and Zfc quantize into the operators 

Q'{zj) = Zj and Q'izj)=C9, (1.16) 

Let dk = ■§^. We compute Ca^{^) = and L^.d^^{^) = \5jk^. This gives 

^'"^^"^^ = ^zMf^) = (^i^ + \^okf) (1-17) 

On half-forms, the observables in q quantize into the operators 

Q'{zjZk) = ZjZk, Q'izjZk) = C^.Q^, Q'izjZk) = jO-q.Cq^ (1.18) 

These operators in (1.16) and (1-18) obey (1.1) and Q'{4')^ = Q'(0) where the inner product 
(/v^b'vA') is again given by the RHS of (1.10). In particular we get 

n 

Q{F) = Ce whereE = J] zfc^fe (1.19) 

k=i 

so that Q{F) is the Lie derivative of the holomorphic Euler vector field on C". 

The operators iQ{(f)), 4> ^ Q-, give a Lie algebra representation of g on iJ by skew-adjoint 
operators. THis integrates to the unitary oscillator representation 

Mp(2m,M) UnitL^„;(C™) (1.20) 

where Mp(2m,M) is the metaplectic group which doubly covers the symplectic group 
Sp{2m,W). This representation splits into exactly two irreducible pieces. 

There is one more thing we can learn from the oscillator example. This is that Kaehler 
polarizations can turn out to be related to cotangent bundle geometry. Indeed, we gave 
no geometric reason for the assignments in (1.16) and (1.18). In quantizing observables on 
cotangent bundles T*Q, we have the guiding philosophy that the principal symbol of Q(0) 
should be if is homogeneous on the fibers of the projection T*Q — > Q. On a Kaehler 
manifold we a priori have no notion like this. 

However, if (M, w) is Kaehler with complex structure J, then we can ask if M is a sym- 
plectic real form of the cotangent bundle T*Z of some complex manifold Z. An obvious choice 
is for Z to be (M, J) (so Z forgets a;). Then the "good" observables on M would be those that 
extend to holomorphic (or maybe rational) functions on T* Z which are homogeneous on the 
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fibers of T*Z — >■ Z. The good observables correspond to bona fide symbols. See §2,3 and [B3] 
for a way to work this out based on the Hamiltonian F . The result of this is easy to describe 
directly for the oscillator. 

We put Z = C". Let Ci;-")Cn be the holomorphic momentum functions on T*Z so 
that 2;i, . . . , Ci, Cn are holomorphic coordinates on T*Z and the canonical holomorphic 
symplectic form on T* Z is = Xlfe=i ^Cfc ^ dz^. Then defines a Poisson bracket {$, 
on the algebra of holomorphic functions on T*Z. We have \^Zj,Zk\Q, = {Cj,Ck}(n = and 
{(j,Zk}n = Sjk- 

We have an obvious complex Poisson algebra isomorphism 

a : C[zi,...,Zn,Zi,...,Zn] C[Z1,.. .,Zn,Cl, -Xn] (1-21) 

where a{zk) = Zk and a(^fe) = iCk- Then a{(p) is the unique extension of to a holomorphic 
function ^ on T*Z with respect to the symplectic embedding 6 of M = C" into T*Z = C^" 
where b{w) = {w,w). Then 

a{zjZk) = ZjZk, a{zjZk) = izjCk, Oi{zjZk) = -QCk (1-22) 

Now the formulas in (1.16) and (1.18) make sense as i(k is the symbol of 

The quantization of the oscillator has manifold applications in physics - in quantum me- 
chanics, quantum field theory, supersymmetry, etc. It also of course occupies a central place 
in mathematics. 

III. Quantization of Hamiltonian Symmetry. To formulate a mathematical quan- 
tization problem generalizing the oscillator case, we suppress (for the time being) the Hamil- 
tonian F and focus instead on the large finite-dimensional symmetry algebra g. This brings 
us to the notion of Hamiltonian symmetry. 

Suppose we have an action of a connected Lie group G on a symplectic manifold (M, a;). 
We regard M as a phase space. Assume the action is symplectic, i.e., G preserves oj. Let g be 
the Lie algebra of G . For each x £ g, wc have the 1-psg (1-parameter subgroup) 7^; : M — > G, 
7a; (i) = exp(te), generated by x. By Nocther's Theorem, there is a smooth function /i^ 
(defined at least locally about every point of M), unique up addition of a constant, such that 
the Hamiltonian fiow of is the action of 7^. Then fi^ is conserved under the action of 7a;. 
If fi^ exists globally on M, then //^ is called a first integral or momentum function for 7^;. 

The symplectic G-action is called Hamiltonian if there exists a map 

: g-^ C°"{M) (1.23) 

X 11^, such that fi^ is a first integral for and = //I^'^'l for all x,y E g, i.e., /x* is 

a Lie algebra homomorphism. Then the functions //^ define a moment map 

li:M^g* (1.24) 

by is^{m) = {ii{m),x). If g is semisimple then we often identify g with its dual by means of 
the Killing form so that moment maps take values in g. 
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The moment map jj, obtained in this way is G-equivariant and Poisson. Consequently the 
image of ^ in g* is a union of coadjoint orbits. The image of the moment map is an important 
invariant of the action. It is easy to prove that is a covering onto a single coadjoint orbit 
if and only if the Hamiltonian action of G on M is transitive; then n is symplectic. Such an 
action is called elementary. 

Thus, symplectically and cquivariantly, the elementary Hamiltonian G-spaces are, up to 
covering, just the coadjoint orbits of G. 

Going back to our oscillator phase space, we see that the action of 5'p(2ra,M) on our 
manifold M = T*R" = C", with the origin of C" deleted, is an elementary Hamiltonian 
action. The moment map C" — {0} — >■ sp(2n, M) is a 2-fold covering on the smallest (non-zero) 
adjoint orbit Or of Sp{2n, M). This orbit Or is stable under scaling and so consists of nilpotent 
elements. 

The quantization problem on the Hamiltonian G-space {M, lo) is to quantize the momen- 
tum functions fi^ into operators in a manner agreeable with Dirac's axioms. It is natural to 
study the elementary case first, as here the symmetry is largest. Thus one seeks a quantization 
of the functions /x^, x G 5, for coadjoint orbits and their covers. 

In analogy with the oscillator, we consider the case where the symmetry group G is a 
real semisimple Lie group Gr (with finite center) and M is an adjoint orbit Or stable under 
scaling. Then Or is a "nilpotent orbit" of Gr - see §2. 

Quantization of coadjoint orbits has traditionally been considered as part of the Orbit 
Method in representation theory. In the Orbit method, one uses polarizations invariant under 
the whole symmetry group and obtains unitary representations by induction. The theory 
incorporates mctaplectic covers and the Mackey machine. Much more can be said about the 
Orbit Method. We note that unitary representations attached to nilpotent orbits are called 
unipotent in representation theory. 

On the other hand, coming into this problem from geometry, we have found different 
methods which apply (at least) to nilpotent orbits. The main idea is to transform the quanti- 
zation problem on Or into a quantization problem on a cotangent bundle, and then solve that 
problem. 

IV. Outline of this Paper. 

In this paper, we quantize the nilpotent orbit Or of Gr in the case where Or is strongly 
minimal (see §3). The oscillator phase space is the double cover of the strongly minimal 
nilpotent orbit of Gr = 5p(2n,M). 

We assume that Qr is simple, the maximal compact subgroup of Gr has finite center, 
and Gr is simply-connected. (Thus we exclude the oscillator there = U{n).) 

We obtain the analogs of the Fock space model of the quantum mechanical oscillator. We 
find analogs of all the features of the oscillator quantization described above in II. This is 
worked out in detail in this paper, with the exception of the integral formula (1.10) for the 
inner product which will be written up elsewhere. We work from scratch and assume no prior 
knowledge on existence of unitary representations. 

This completes the work from [B-K4]. In [B-K4] we worked out with Kostant the results 
covered in §4-7 of this paper for the three cases where Gr is a a split group of type E^, Ey, 
Es- 
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We start from the fact, a product of the work of Kronheimer ([Kr]) and Vcrgne ([Ve]), that 
Or admits a JCR-invariant complex structure J which together with the KKS symplectic form a 
gives a (positive) Kaehler structure on Or. The Vergne diffeomorphism V : Or Y identifies 
the complex manifold (Or, J) with a complex homogeneous space Y of the complexification 
K of K-g^. This is a general theory that applies to every nilpotent orbit for G-g. semisimple. 
For the oscillator, this recovers the C/(n) -invariant Kaehler structure and the identification 
r*]R'^ = C" used in II. 

We outline this theory in §2 and we explain how it gives rise to an embedding of Or 
into T*Y as a totally real symplectic submanifold ([Bl]). This enables us to transform the 
quantization problem on Or into a quantization problem on T*Y, as long as the Hamiltonian 
functions 4>'^ , w £ gg^ extend from Or to T*Y. 

An important aspect is that the Kaehler structure on Ok possesses a global Kaehler 
potential p which we argue plays the role of the Hamiltonian F. The Hamiltonian fiow of p is 
the action of the center of Kg in the oscillator case. In our cases, the Hamiltonian flow of p 
lies outside the GR-action. 

In §3, we specialize to the case where Or is strongly minimal and Kg has finite center. We 
explain how to convert the Hamiltonian functions (p"^ , w £ Qr, on Ok into rational meromorphic 
functions on the cotangent bundle of Y. We interpret the as "pseudo-differential 
symbols" . 

To describe the symbols, we consider the Cartan decomposition 5r = ^r © pR (cf. (1.13)). 

For X G ^K, is just the usual symbol of the holomorphic vector field r]^ on Y defined by 
differentiating the i^-action. But for v G pK, is a sum of two terms, each homogeneous 
under the fiberwise scaling action of C* on the leaves of the cotangent polarization of T*Y . 
The passage from the observable function ^™ to the symbol preserves Poisson brackets. 

The middle part §4-§7 of the paper is devoted to quantizing the symbols w G 0k, 
into skew-adjoint operators on a holomorphic half-form line bundle over Y. In §5, we 
construct all such bundles. We find the space H of global algebraic holomorphic sections of 
N2 is a multiplicity free ladder representation of K. We get a simple geometric description of 
the sections which are the highest weight vectors. 

In §4, we set up the Jordan structure that is used throughout the paper (explicity in §5 
and §7). A main point is that the polynomial function P constructed in §3 is realized in terms 
of Jordan norms. 

We construct, in Corollary 6.2 and Theorem 6.3 the pseudo-differential operators Q{^^) 
on half-forms which quantize the symbols or equivalently, the functions (f)'^. Theorem 6.3 
says that these operators satisfy (1.1), i.e., the operators tt'^ = ?Q($"') give a representation 
of g. In Theorem 6.6 we construct the gK-invariant inner product B on H. In Theorem 6.8 
we compute B by giving the analog (6.30) of (1.9). 

Our operators are pseudo-differential (not purely differential) in that they involve inverting 
the positive-spectrum "energy" operator E' which is the quantization of p. In fact, instead of 
the order two operators Lq.Lq^ from (1.18) we obtain order 4 differential operators divided 
by E' {E' + 1); these are "formally" of order 2. The action of the maximal compact group Kg 
on H is just the natural one defined by the action of Kg on Y and N2 . 

Theorem 6.4 says that our representation tt of g on is irreducible. Also we describe the 
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algebra generated by the operators tt"' on H. It follows in Theorem 6.6 that tt integrates to 
give an irreducible minimal unitary representation of Gk on the Hilbert space completion H 
oiH. 

Next §7 is devoted to proving the results of §6. We show that our pseudo-diflferential 
operators satisfy the bracket relations of 0k by reformulating the problem and applying the 
generalized Capelli Identity of Kostant and Sahi ([K-S]). An important aspect of their work 
is that Jordan algebras provide a natural setting for generalizing the classical Capelli identity 
involving square matrices. The complex Jordan algebra t-i occurring here is semisimple 
(while in [B-K4] it was simple). It turns out that the simple components of €_i become 
coupled together in our calculations in a subtle way reflected by Proposition 7.8. 

In §8, we compute the reproducing kernel /C of the Hilbert space completion 7i of H. We 
find that /C is a holomorphic function on y x y and hence Ti consists entirely of holomorphic 
sections of N^. Finally, in §9 we give some examples. 

Diflferent models, or proofs of existence, for most of the unitary representations we con- 
struct have been obtained by other authors. These include Binegar, Gross, Howe, Kazhdan, 
Kostant, Li, Oersted, Rawnsley, Savin, Sijacki, Sternberg, Sabourin, Torasso, Vogan, Wallach, 
Wolf, and Zierau. Moreover in [T], Torasso constructs in a uniform manner by the Orbit 
Method Schroedinger type models of all minimal unitary representations. Precisely, Torasso 
constructs unitary irreducible representations attached to all minimal admissible nilpotent or- 
bits of simple groups of relative rank at least three over a local field of zero characteristic. It 
would be very interesting to construct intertwining operators between our models. 

There is a rich litcratTire on geometric models of unitary highest weight representations, 
and there are many interesting ties here with our work. 

This paper builds on several years of joint work with Bert Kostant on the algebraic holo- 
morphic symplectic geometry of nilpotent orbits of a complex semisimple Lie group. This work 
includes [B-Kl-5]. In addition §4 of this paper is joint work. 

I thank Alex Astashkevich, Olivier Biquard, Murat Gunaydin, Bert Kostant, Michele 
Vergne, and Francois Ziegler for useful conversations relating to this work. Parts of this work 
were carried out during visits to Harvard (1993-94, summers of 1995 and 1996), the Institute 
for Advanced Study (Spring 1995) and Brown University (summer of 1997). I thank all these 
departments for their hospitality. I thank Mark Gotay for putting together this volume and 
for his comments on my paper. 

I am delighted to dedicate this paper to Victor Guillemin and to be able to contribute it 
to this volume in his honor. In my graduate student days at MIT I was ensconced in algebraic 
geometry and algebraic group actions. I was symplectically agnostic. But since my symplectic 
conversion in the end of the last decade, I have had the opportunity to talk to Victor a lot 
and learn from him and his many books and papers. I thank him for warmly welcoming me 
as a visitor into his symplectic group. 

§2. The Quantization Problem for Real Nilpotent Orbits. 

The phase spaces we wish to quantize are the so-called "nilpotent orbits" of Gk where 
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is a connected non-compact real semisimplc Lie group with finite center. Then G^. is a finite 
cover of the adjoint group of its Lie algebra g^, and 0k is semisimple. To define the nilpotent 
orbits we consider the coadjoint action of on the dual of Qm.- 

Each coadjoint orbit Ok carries a natural GiR-invariant symplectic form a, often called the 
KKS or Lie-Poisson form. The form a is uniquely characterized by the following property: let 

(I>:5r^C'^{Om), (2.1) 

be the puUback map on functions defined by the embedding Or C g^. Then ^ is a Lie algebra 
homomorphism with respect to Poisson bracket on C°°(Or) defined by a. 

In analogy with the cotangent bundle, we wish to single out those coadjoint orbits which 
are conical in the sense that they are stable under the Euler scaling action of M+ (positive 
reals). There is a nice Lie theoretic characterization of these orbits. To get this, we first use 
the Killing form ( , )gR to identify with gg; we do this throughout the paper routinely. Then 
(conical) coadjoint orbits get identified with (conical) adjoint orbits. 

An adjoint orbit is conical if and only if it consists of nilpotent elements in g^. Such orbits 
are called "nilpotent orbits" . It is well-known in Lie theory that there are only finitely many 
nilpotent orbits in gR. 

Prom now on, we take Or to be a nilpotent orbit in gR. The quantization problem on 
Or is to quantize into operators the functions G g^. This is a reasonable goal. Ideally 

quantization would convert all smooth functions on Or into operators in a manner satisfying 
Dirac's axioms. See, e.g., [Ki,§2.1] for a complete axiom list. But full quantization is impossible 
even for polynomial functions on (the infamous No-Go Theorem). We are left hoping that, 
except for anomalies, finite-dimensional Hamiltonian symmetry will quantize. 

In analogy with the Pock space quantization of the oscillator, we look for a Kaehler polar- 
ization of our phase space (Ok, a) which is invariant under a fixed maximal compact subgroup 
i^R of Or. This means that we look for a iCR-invariant integrable complex structure J on Or 
such that J and a together give a (positive) Kaehler structure on Or. 

Portunately, such a complex structure J on Or arises from the works of Kronheimer ([Kr]) 
and Vergne ([Ve]) on instantons and nilpotent orbits. This gives the KK-invariant instanton 
Kaehler structure (J,(t) on Or. This structure is discussed and studied in detail in [Bl]. We 
recall two main points. 

The first point is the Vergne diffeomorphism ([Ve]). To set this up, we introduce the 
Cartan decomposition 

gE = «M ® PM (2.2) 

where Ir C gR is the Lie algebra of K-^. and pR is its orthogonal complement with respect to 
the Killing form. The natural action of on pK complexifies to a complex algebraic action 
of on p where K is the complexification of Kg. (so that is a complex reductive algebraic 
group) and p = pK ® ipR. 

Now the Vergne diffeomorphism 

V : Or ^ F (2.3) 
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is a {Ks. X M"'")-equivariant diffeomorphism of real manifolds which maps Ok onto a AT-orbit Y 
in p. Y, being a i^-orbit, is manifestly a complex submanifold of p. Moreover J is the pullback 
through V of the complex structure on Y. 

An important feature is that Y is stable under the Euler scaling action of C* on p. This 
follows since Or is M^-stable and V is M^-equivariant. Let E be the infinitesimal generator of 
the Euler C*-action so that E is the algebraic holomorphic Euler vector field on Y. 

In general, the target Y of the Vcrgne diffeomorphism is kno\¥n (by the Kostant-Sekiguchi 
correspondence [Sek]) but not the actual map giving V. A little insight into V comes from Lie 
theory. 

To explain this, we introduce the complexified Lie algebra = 0r © i0R. g is a complex 

semisimple Lie algebra and carries the complex conjugation map x + iy x + iy = x ~ iy. 

An S-triple in g is a basis (e, h, f) of a subalgebra isomorphic to sl(2, C) which satisfies the 
bracket relations [h, e] = 2e, [h, /] = —2/, [e, /] = h. The S-triple is adapted to (ga, Jr) if e 
and / are complex conjugates and h G it^. Given Or, we can find an S-triple (e, h,e) adapted 
to (gM, ^k) such that e + ih + e lies in Or. Then Vergne's construction gives 



The second point is that the Kaehler structure (J,cr) on Or admits a global Kaehler 

potential p. This means that p is a smooth real valued function on Or such that iddp = oj. 
Moreover p is uniquely determined by the added condition that p transforms homogeneously 
under the Euler M"*" -action on Or. Then p is iCR-invariant and Euler homogeneous of degree 
1. 

Next we examine how to use this Kaehler structure in quantization. The Vergne diffeomor- 
phism identifies Y as Or equipped with a complex polarization. The philosophy of Geometric 
Quantization now predicts that we can quantize suitably nice real-valued functions (p on Or 
into self-adjoint operators on a Hilbert space consisting of holomorphic sections of a suitable 
holomorphic vector bundle over Y. 

Our quantization program for Or becomes: "quantize" each function (j)'^ , w E gR, into a 
self-adjoint operator Q{4>'^) on a Hilbert space TC of square integrable holomorphic sections of 
a holomorphic half-form complex line bundle Ni over Y in such a way that the Dirac axiom 



is satisfied. In the course of doing this, we will end up quantizing one additional function on 



There are additional axioms which should also be satisfied, but these are somewhat hidden 
as we are only dealing with the functions cf)"^ . E.g., the axiom that the constant function 1 
quantizes to the identity operator is "hidden" . These "hidden axioms" are basically incorpo- 
rated by our methodology developed below using symbols. 

If the Hamiltonian flow of (f) preserves J and ^ is homogeneous of degree 1, then we 
mandate that the quantized operator is simply 



V{e + ih + e) = e 



(2.4) 



Or. 



(2.5) 
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Here is the 3-Hamiltonian vector field on Y defined by the condition that is holomorphic 
and coincides with on holomorphic functions. We write £^ for the Lie derivative operator 
(acting on holomorphic half-forms) with respect to a holomorphic vector field rj. 

Diflferentiating the i^^-action on Y we get an infinitesimal holomorphic vector field action 

t mct^°^ Y, x^ri'' (2.6) 

Then r]^ = ^^f^x for x & tu. and so 

Qicj)^) = -iCr,^, forxefiE (2.7) 

The problem, since our polarization J is only J^K-invariant, is to quantize the remaining 
functions (f>'", v E pM. corresponding to the second piece in the Cartan decomposition (2.2). 

A key aspect of our program for quantization of real nilpotent orbits (see [Bl-3]) is that we 
regard p as the Hamiltonian function on Or. This generalizes the case of the harmonic oscillator 
discussed in §1 where the Hamiltonian is the total energy. It may seem strange that the 
oscillator energy Hamiltonian is homogeneous quadratic while our function p is homogeneous 
linear. However the oscillator phase space M^" — {0} arises as the double cover of a real nilpotent 
orbit. In that case, our linear potential function p does indeed pull back to a quadratic function 
on M^"^ — {0}, and it is easy to check that we recover the classical energy Pi + qi+ - • • +Pn + Qn 
(see [B3]). 

In physical terms, the Hamiltonian governs the time evolution of the classical system. 
The quantum mechanical problem is to find the eigenvalues and eigenstates of the operator 
quantizing the Hamiltonian. 

Thus we now demand that quantization should not only promote the symmetry functions 
(j)'^ to operators, but should also promote p to an operator. In fact the Hamiltonian fiow of p 
preserves J and is periodic; we call this the KV (Kronheimer-Vergne) S'^-action on O® ([Bl]). 
Under V, the KV S'^-action corresponds to the circle part of the Euler C*-action on Y. It 
follows that the J-Hamiltonian vector field of p is iE. Hence 

Q{p) = -iCiE = Ce (2.8) 

Let VL be the canonical holomorphic symplectic form on T*Y . Then Vl defines a Poisson 
bracket on the algebra of holomorphic functions on T*Y , and also on the field of meromorphic 
functions. 

A main result of [Bl] is to realize the holomorphic cotangent bundle (T*y, O) as a sym- 
plectic complexification of Or. To do this, we push forward p to a smooth function py on Y 
so that p = py oV. Next we construct the following real 1-form /3 on Y" 

l3 = -\{,d-d)pY (2.9) 
Then j3 defines a smooth section of the cotangent bundle T*Y ^Y. 



12 



Theorem 2.1 [Bl]. The composition 

6 : Or F ^ T*Y (2.10) 

embeds Ojr as a totally real symplectic submanifold ofT*Y. In particular, 6*(ReO) = a and 
6*(ImQ) = 0. 

Now, given a function (f) on Ok which we wish to quantize, we can ask if ^ extends to a 
holomorphic function $ on T*Y. (Such an extension, if it exists, is necessarily unique.) If so, 
then $ is our candidate for the symbol of Q{(/)). 

This philosophy is consistent with what we already found in (2.7) and (2.8). Indeed we 
can define the holomorphic symbols, where x £ t, 

= symbol rj"^ and A = symbol E (2.11) 

Our convention for symbols is specified by the following formula in holomorphic Darboux 
coordinates: 

f)koA \-km 

symbol f{zo, . . . , ^™)— — ^ = /(^o, ■ • • , Zm)i''°+-+'-Co' ' ' ' Ct 

It is easy to check ([Bl,3]) 
Corollary 2.2. 

(i) The Kaehler potential p on Ok extends uniquely to a holomorphic function on T*Y. Pre- 
cisely, p extends to \ = symbol Q{p)- 

(ii) For x G 6r, (f)'^ extends uniquely to a holomorphic function on T*Y. Precisely, (f)^ extends 
to = symbol Q(^^). 

In effect, (3 was engineered to make (i) true. 

The passage from functions on Or to holomorphic functions on T*Y preserves Poisson 
brackets. I.e., if and $2 are respectively the holomorphic extensions of two real functions 
and (f)2 on Ok then {#1, #2}n is the holomorphic extension of {(t>i-,(t>2}cT, where the subscripts 
indicate the symplectic form defining the Poisson brackets. This follows easily from Theorem 
2.1. 

Thus if all the Hamiltonian functions (f>'^ , w £ g^, extend holomorphically from Or to T*Y, 
then this in eff^ect converts our quantization problem on Ok into a holomorphic quantization 
problem on r*y. 

This brings us to the question as to whether the Hamiltonian functions 4>'" , v E pm., extend 
to holomorphic functions on T*Y. The general answer is no. However, the better question 
is whether the (j)'" extend to meromorphic functions on T*Y. In [B3], we show that the 
answer is yes in every case, at least if we allow the to lie in a finite extension of the field 
of meromorphic functions on T*Y. This relies on the powerful result of Biquard [Bil, Bi2] 
that the homogeneous hyperkaehler potential on a complex nilpotent orbit is always a positive 
Nash function. 
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In fact, the symbols that arise here are all rational functions on T*Y (or at least regular 
functions on an etale cover of a Zariski open set of T*Y) in the sense of algebraic geometry. The 
holomorphic symplectic form O is manifestly algebraic and so Q defines a Poisson bracket on 
the algebra R{T*Y) of algebraic holomorphic functions on T*Y and also on the field <C{T*Y) 
of rational functions on T*Y . 

In the next section, we explain in detail how this works for the smallest orbits. 

§3. Pseudo-Differential Symbol Realization of gjR for Ok Strongly Minimal. 

Each nilpotent orbit Or C lies in a unique complex adjoint orbit O C g. Then O 
is a complex nilpotent orbit (i.e., O consists of nilpotent elements in g). We call O the 
complexification of Or. The nilpotent elements in g are characterized by the property that 
their adjoint orbits are stable under the scaling action of C*. 

We assume from now on that the complex Lie algebra g is simple. Let G be the adjoint 
group of g. Then G is a connected complex semisimple algebraic group with Lie algebra g and 
complex conjugation on g defines a complex conjugation map g 'g on G. Let ( , )g be the 
complex Killing form of g. We often identify g with g* and 6 with t* by means of ( , )g. 

Recall from §2 that K-^. C Or is a fixed maximal compact subgroup with complexification 
K. We have natural maps Ok G and K ^ G and both maps have finite kernel. 

Since g is a simple Lie algebra, the adjoint representation of O on g is irreducible. The 
orbit Oniin of highest weight vectors is then nilpotent, as it is the orbit of a highest root vector. 
Moreover, Omin is minimal among all non-zero nilpotent orbits in the sense that it lies in the 
closure of every non-zero nilpotent orbit. It follows that Omin is the unique (non-zero) minimal 
nilpotent orbit. 

We will call a real nilpotent orbit Ok strongly minimal if the complexification of Ok is 
Oniin- In Theorem 4.1 below we recall from [B-K5] the classification of strongly minimal real 
nilpotent orbits. For Ok strongly minimal, formulas for V and p are easy to write down because 
the action of K-g^ x R+ is transitive on Or. (However, for general Or the action is not transitive, 
and working out V and p is a hard open problem.) 

As g is simple, there are just two possibilities for the center of Kf^: either (i) Cent K^i is a 
circle subgroup or (ii) Cent is finite. These cases correspond exactly to the nature of the 
irreducible symmetric space G^/K-g^, so that G^/Kg is Hermitian in (i) and non-Hermitian in 
(ii). Accordingly, we call the complex symmetric pair (g,l) Hermitian or non-Hermitian. 

For each u G p, let be the linear function on p defined by fv{u) = {v,u)q. Then by 
restriction to Y we get a iiT-equivariant complex linear map 

p ^ R{Y), v^U (3.1) 

Every algebraic holomorphic function on Y defines an algebraic holomorphic function on T*Y 
by pullback through the projection T*Y Y. 

From now on in §3, we assume that Or is strongly minimal and the center of Kg is finite. 

Theorem 3.1. Let v & pg and x & ig. 
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(i) Recall the embedding b : Or — > T*Y from (2.10). Each function (jf on Or extends uniquely 
to a rational function on T*Y . Set #^+^ = + where #^ was defined in (2.11). 
The resulting linear map 

0K ^ C{T*Y), w^^^ (3.2) 

is a 1-to-l real Lie algebra homomorphism with respect to the Poisson bracket on 'C{T*Y) 
defined by Q.. 

(ii) Each rational function is everywhere defined on the Zariski open dense complex alge- 
braic submanifold 

M = {m € T*Y I A(m) ^ 0} (3.3) 

so that is algebraic holomorphic on M. 

(iii) We have 

= 5. (3.4) 

where g^ is an algebraic holomorphic function on M which is homogeneous of degree 2 
with respect to the Euler C* -action on the fibers of the the projection M ^ T*Y Y. 

Proof. This is proven in a more general setting in [A-Bl]. ■ 

We write R{X) for the algebra of algebraic holomorphic functions on a complex algebraic 
variety X. Recall from (2.11) that A G R{T*Y) is the symbol of the Euler vector field. 

Lemma 3.2. We have R{M) = R{T*Y)[X-^]. 

Proof. This follows easily since M is the complement of the irreducible divisor (A = 0) in the 
smooth (and hence normal) variety T*Y. M 

It is natural now to extend (3.2) C-linearly so that = $^ + i^v for x,y e Qr. This 

is consistent with (2.11). We have the complexified Cartan decomposition 

g = fi®p (3.5) 
Corollary 3.3. The map (3.2) extends to a 1-to-l complex Lie algebra homomorphism 

0^ ii;(r*F)[A-^], z^^^ (3.6) 

Then for v e p we have again the same formula (3.4). 

The significance of Corollary 3.3 is that we can regard functions in R{T*Y)[X~^] as 
"pseudo-differential" symbols; cf. §6. 
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We have now, in Theorem 3.1 and Corollary 3.3, transformed our original problem of 
quantizing the functions 0'", w G gjj, on O^. into the problem of quantizing the rational 
functions u> G gjR, on T*Y. We mandate 

Q($"') = Qir) 

The new problem lies in the holomorphic symplectic category: the problem is to quantize each 

into a self-adjoint operator Q{^'^) on a Hilbert space consisting of holomorphic sections 
of a holomorphic half-form bundle on Y. 

The advantage of the new problem is that <I>™ is already a symbol, and so we can try to 
quantize it by constructing reasonable quotients of differential operators with symbol We 
emphasize that (3.4) says that , v & pR, is not a principal symbol, but instead is a sum of 
two principal symbols fy and gy. We will get around this by a naive trick: we will quantize 
fy and Qy separately and then add the answers. 

Since fy is just a holomorphic function on Y, we mandate that the quantization of /„ is 
Q{fy) = fy, i.e., Q{fy) Is the operator defined by multiplication by fy. 

The aim of the rest of this section is to state a formula for the symbols gy. We want to 
express gy in terms of the basic symbols fy, v e p, x e t, and A since we already know how 
to quantize these symbols. To work this out, we construct a set of local (etale) coordinates on 
T*Y consisting of basic symbols in Lemma 3.5 below. 

We begin by setting up some of the Lie theoretic structure associated to Or following 
[B-K4,§2]. We will make use of this throughout the paper. We note that the discussion of 
Omin in [B-K4,§2] was in the same generality we have here, and it was only from §3 onwards 
in that paper that the work specialized to the three cases where Gu is split of type Eq, or 
Es- 

To begin with we have 

Or = Omin nQR = Gk- (e + ih + e) and Y = O^in np = K -e (3.7) 

where (e,/i,e) are chosen as in §2. Then 

s = Ce e e Ce (3.8) 

is the corresponding sl(2, C)-subalgebra. We assume from now on that ( , )g is rescaled so that 
(e,e)g = 1. 

The action of ad/t on g is diagonalizable with spectrum {±2, ±1,0} so that we have the 
5-grading 

= 02 01 ©00 ©0-1 ©0-2 (3.9) 
where the subscripts indicate the corresponding eigenvalues. Then 0s = © Ps and 

t = ii®io®i-i and p = p2 ©Pi ©Po © P-i © P-2 (3.10) 
Clearly then i±i and p±i are abelian Lie subalgebras. We recall from [B-K4, §2. 2.-2.4] 
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Lemma 3.4. The spaces g±2 are 1- dimensional with 

92 = p2 = Ce and fl_2 = p-2 = Ce (3.11) 

We have dimcps = dimcp-s and dimc^s = dimc^-s- The Lie bracket defines a perfect 
pairing 6i x pi ^ Ce. Thus we may define 

m = dime p±i = dime (3-12) 

The subspace 02 ©Si is a {2m + 1)- dimensional Heisenberg Lie algebra with center 52- We 
have = 0*^© Ch © 02 © fli • Consequently 

dime Oniin = 2m + 2 (3.13) 



In particular 



dime Y = ^ dime Omin = m+l (3.14) 



Now we take a basis vi, . . . ,Vm of pi. We put vq = e. The corresponding regular functions 
on Y defined by (3.1) are 

/o = fvoj /l = fvn . . . ) fm = fvm (3.15) 

These form a system of local coordinates on Y by ([B-K3, Prop. 5.2]). In fact we get an 
isomorphism of varieties 

yo ^ C* X y ^ (/o(y), h{y), fm{y)) (3.16) 

where c Y" is the open set given by 

F° = (/o 0) (3.17) 

In our local coordinates we have 

Let {xi, . . . ,Xjn} be the basis of t\ such that [a;j,Vj] = 5ije. Then in terms of our local 
coordinates fo, fi, ■ ■ ■ , fm the expressions for our vector fields E and rj^*, i = 1, . . . , m are 

It follows easily from these formulas that 
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Lemma 3.5. The 2m + 2 functions /o, /i, ...,/„, A, , . . . , form a system of local etale 
coordinates onT*Y. 

The single function g^^ determines all the functions because of the i^-action; indeed, 
9lx,v] = {^'^,gv}n for X G t. To state our formula for g^^^ we need one more ingredient, the 
polynomial function function P defined below. 

The vector fields r]^, x e i define a natural complex algebra homomorphism from the 
universal enveloping algebra U{t) to the algebra ^^{Y) of algebraic holomorphic differential 
operators on Y. So in particular we get a representation 

7TK ■■ U{t) -> End R{Y) (3.20) 

On the symbol level, (3.20) corresponds to the graded Poisson algebra homomorphism 

■■ S{t) R{T*Y) (3.21) 

defined by ^k{^) = ^'^ for x € I. 

The adjoint action of g defines a a complex algebra homomorphism ad :lA{t) — ^ Endp, 
Q ^ &dQ = adg. Let P be the polynomial function on t-i defined by 

ladj(e) = P{y)e (3.22) 

where y G Then P is homogeneous of degree 4. 
We have a perfect pairing 

ti X «_i ^ C (3.23) 

defined by the Killing form ( , )g as in [B-K4,§2.5]. This gives an identification of S{ii) with 
the algebra of polynomial functions on This identification places P G >S''*(6i) so that we 
may write 

P = P(xi,...,x„) and = P(^''S ) (3.24) 

Theorem 3.6. The function g^ in Theorem 3.1(iii) is given for v = vq = e by 

5.0 = -T^^ (3.25) 



Proof. A more general result is proven in [A-B2]. ■ 

In the next section, we set up the Jordan algebra machinery which gives us a useful and 
computable way to understand the polynomial P. In §5, we already use this machinery to 
classify half-form bundles on Y. The reader eager to see how we quantize the symbols g^ and 
then the symbols can skip ahead to Lemma 5.3 and Proposition 5.5 and then to §6. 
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\4. Complex Minimal Nilpotent Orbits and Jordan Algebras. 



Our first aim in this section is to classify the real simple Lie algebras Qr which possess a 
strongly minimal real nilpotent orbit. This amounts to classifying Qm. such that Omin has real 
points because of (3.7). This classification, recalled in Theorem 4.1 below, uses the geometry 

of Omin- 

Any complex nilpotent orbit O, and so in particular Omin; is a quasi-afiine smooth locally 
closed complex algebraic subvariety in g. This follows since the adjoint action of G on g 
is complex algebraic. Furthermore, O is an algebraic holomorphic symplectic manifold with 
respect to its G-invariant holomorphic KKS symplectic form S (cf. [B-Kl]). The G-action on 
O is Hamiltonian with holomorphic moment map given by the embedding O C g. 

Let fj, : g ^ i he the projection defined by (3.5). Then the composite map fj, : Omin 
Q ^ t is the moment map for the Hamiltonian if -action on Omin- Let J\f{t) be the cone of 
nilpotent elements in t. 

Theorem 4.1[B-K5]. The following conditions are equivalent: 

(i) Omin n gR is empty 

(ii) K has a Zariski open orbit on Omin 

(iii) Ai(Omin) C AA(«) 

and imply that the principal isotropy group of K on Omin is where s was defined in (3.8. 
The complete list of all complex symmetric pairs (g, 6) {with g simple ) which satisfy (i)-(iii) 

is: 

(a) (s[(2n,C),sp(2n,C)), where n>2 

(b) (5o(p + l,C),so(p,C)), where p> 3 

(c) (5p(2j9 + 2g,C),sp(2p,C) +sp(2g,C)), wherep,q>l 

(d) (F4,so(9,C)) 

(e) {Ee,F^) 

Each pair (g,€) in this list is non-Hermitian. 

From the point of view of representation theory, the condition that Omin H gR is non-empty 
is very natural. To explain this, we recall the theory of the associated variety. 

Suppose TTo : Or UnitH is an irreducible unitary representation. Let H C H he the 
space of i^K-finite vectors with its natural (g, if)-module structure; H is then the Harish- 
Chandra module of the representation. Differentiation of the group representation gives a Lie 
algebra representation of gR on H and so a representation n ■.U{g) End H of the universal 
enveloping algebra. The annihilator I is then the primitive ideal attached to tTq. The graded 
ideal grX cuts out a closed complex algebraic subvariety V(grX) C g* — g called the associated 
variety of X. Since tTq admits a central character, it follows that V(grT) is a union of complex 
nilpotent orbits. A basic result (due independently to Borho and J.L. Brylinski, to Ginzburg, 
and to Joseph) is that V(gr J) is in fact the closure of a single nilpotent orbit, which is then 
called the associated complex nilpotent orbit of H and H. 

The following observation is an easy consequence of the theory of associated varieties. For 
instance, it is a corollary of [Vo2, Theorem 8.4]. 
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Lemma 4.2. Suppose Gr admits an irreducible unitary representation with associated complex 
nilpotent orbit O C g. Then O Dqm. is non-empty, i.e., O has a real form with respect to 0r. 

We will call an irreducible unitary representation tTq : Gr — > Unit H minimal if its associ- 
ated nilpotent orbit is Omin and also the image of vf : hl{Q) — > End H has no zero-divisors , i.e., 
the annihilator X of vr is completely prime. For g not of type tTq is minimal if and only 
if X is the Joseph ideal. So Lemma 4.2 says that a necessary (but not sufficient) geometric 
requirement for to admit a minimal representation is that Omin H 0m is non-empty. 

From now on in this paper, we assume that Or is strongly minimal and (g,fi) is non- 
hermitian. For convenience, we also take Or to be simply-connected. There is no problem in 
this as the universal cover has finite center. 

We freely identify Or with Y via the Vergne diffeomorphism (2.3). Using Lemma 3.4 and 
(3.14), we find the dimension of Y is given by: 

Q 5l(n,C) 50(n,C) sp(2n,C) G2 F4 E7 Es , . 

dime y n-1 n-3 n 3 8 11 17 29 ^ ' 

Next we want to develop the Jordan theory interpretation of the polynomial P defined in 
(3.22). We find in Proposition 4.4 below a Jordan structure on the space 6_i from (3.10). 

There is a natural symmetry group acting on the space namely the isotropy group 

= for the adjoint action of K. We use this symmetry throughout the paper. Kq is a 
closed reductive complex algebraic subgroup of K with Lie algebra 60 • Also Kq is connected; 
this follows immediately from the fact that the adjoint orbit K ■ h is simply-connected. Basic 
constructions like (3.10), (3.16), and (3.17) break the if -symmetry but not the if^o-symmetry. 

In particular, Kq acts on p2 by a (non-trivial) character 

X-Ko^C* (4.2) 

so that a • e = x(a)e for a G Kq. Let 

K'o = kernel of x = (4.3) 
and let = be the Lie algebra of Kq. We get an orthogonal decomposition 

€0 = ^0 e (4.4) 
Since is abelian, we have a natural identification 

U{h) = S{h) (4.5) 

So in particular, P defines an element of ^(61). We recall from [B-K4,§2.2-2.6]: 

Proposition 4.3. The polynomial P £ <S"^(6i) defined in (3.22) is semi-invariant under 
Kq and transforms by the character ■ Moreover, P is, up to scaling, the unique K^-semi- 
invariant polynomial in S{ii) such that 

adp(Ce) = Ce (4.6) 
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We recall some work from [B-K4, §2.6-8]. We found with Kostant a nilpotent element 
ee G ti such that 

[ = C/t ® Get e Cee (4.7) 

is a complex Lie subalgcbra in 6 isomorphic to sl(2,C) and {2h, — ef,ee) is an S-triple basis of 
I. We normalized the choice of ee so that ^(adet)^(e) = e and ^(ade{)^(e) = e. Hence 

P(ee) = 1 (4.8) 

We note that the nilpotents e € p and et G were called, respectively, z and e in [B-K4]. 

Then we showed that (6, 6o) is a Hermitian symmetric pair of tube type with rank q where 
q < A. In addition, the pair (toi^^) is a complex symmetric pair so that we have a complex 
Cartan decomposition to = (Bt where [t, t] C t'. 

Consequently, elaborating on [B-K4, Proposition 2.8] we get 

Proposition 4.4. The Tits-Kantor-Koecher construction gives i-i the structure of a complex 
semisimple Jordan algebra with -invariant Jordan product defined by 

[x, et] o [y, ej] = [x, [y, e^]] (4.9) 

where x,y e t. The Jordan identity element is e^. The Jordan algebra degree o/6_i is 

degt-i = q = rank {t, to) < 4: (4.10) 



The T-K-K theory identifies t-i as the complexification ^Tc of a real Euclidean Jordan 
algebra J^. The book [F-K] is an excellent reference for the theory of real Euclidean and 
complex semisimple Jordan algebras. 

Next we write out the decomposition of t_i into a direct sum of complex simple Jordan 
subalgebras: 

i-i=)[i]® (4.11) 

Then each space carries an irreducible representation of to- Let qn be the degree of )[„]; 
then 

gi + ••• + © = g (4.12) 
Let P[„] be the Jordan norm of )[„]; then P[„] has degree qn- 

Proposition 4.5. The polynomial P G S^{ti), constructed in (3.22), as a function on t-\, is 
uniquely expressible as a monomial 



P _ T3W1 pW, 
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(4.13) 



in the Jordan norms P[„] of the simple components ofl-i. Every exponent wi,. . . ,Wi is 
positive. 

Proof. In the Tits-Kantor-Koecher construction, the simple Lie components of i correspond 
to the simple Jordan components of I.e., = 6_i fl r[„] where t = r[i] © • • • © r[£] is the 
decomposition of t into complex simple Lie subalgebras. Then we get also the decomposition 
60 = t[i] © • • • © t[f] into a direct sum of subalgebras where t[„] = to H r[„] . Each pair (r[„] , t[„] ) 
is complex symmetric of tube type. Then any polynomial function on semi-invariant 
under t[„] is a polynomial in see, e.g. [K-S, Th. 0]. It follows that P is of the form 
P = cPjYj^ • • • Pj^j^ for some scalar c. But c = 1 since -P(ee) = P[i](ej) = • • • = P[^](e{) = 1. 
Finally the fact that h has non-zero projection to each component t[i], . . . ,t[^] implies that 
each wi,... ,we is non-zero. ■ 

Notice that (4.13) defines the exponents wi,. . . ,we and gives the numerical equality 

qiwi -\ qiwe = 4 (4.14) 

In [B-K4], we quantized with Kostant the real form Or of Omin in the three cases where 
Jc is a simple complex Jordan algebra of degree 4 so that £ = 1 and P = P[i] . In this paper 

we treat the general case where P may factor non-trivially. 

We proceed in the rest of this section to make an explicit list of the Jordan algebras 
occurring here. The associations we get between exceptional Lie algebras and Jordan algebras 
are in many cases already familiar from the constructions discovered by Tits, Kantor, Koecher, 
and Allison and Falkner to produce exceptional Lie algebras out of Jordan algebras. 

In our tables we adopt the following conventions. We write sOp, sp2p, sip, G2, P4, Eq, 
E7, Es for the corresponding complex Lie algebras. Also S^C^ denotes the irreducible repre- 
sentation of 5o(p,C) satisfying S^C^ + S'^'^Cp ~ S'^C^ while AJJC^p denotes the irreducible 
representation of sp(2p, C) satisfying A'^C'^p + A"'~^C^p c± A"C^p where n < p. 

A complete list of all non-isomorphic formally real simple Jordan algebras of degree < 4 
follows immediately from the Tits-Kantor-Koecher theory and the known list of all irreducible 
Hermitian symmetric tube domains (see [H], p. 528, Example 4 and §6.4, pp. 518-520). We 
give this list in Table 4.6 together with the corresponding pair (t, to)- The number d arises in 
the following way. The restricted root system for the pair (t,to) is of type Cq where q is the 
degree of J and then the long roots have multiplicity 1 while the short roots all have common 
multiplicity d. 

In Table 4.6, W, p > 1, is the p-dimensional real Jordan algebra associated to the Eu- 
clidean norm and Herm(n, F) is the real Jordan algebra of n x n hermitian matrices over F 
where H and O denote the quaternions and the octonions (the Cayley numbers) respectively. 
Then Herm(3, 0) is the exceptional 27-dimensional Jordan algebra while all the others in Table 
4.6 are special (i.e., arise in the standard way from associative algebras). The last column in 
Table 4.6 gives a name to the Jordan norm of J". 

Table 4.6. All Simple Euclidean Real Jordan Algebras of rank < 4 
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In Table 4.7 we list all pairs (g,t) occurring here (i.e., non-hermitian complex symmetric 
pairs (9,^) with Omin H 9r 7^ where 9 is simple) together with the Jordan algebra J arising 
from the T-K-K theory and the polynomial function P G 5"^ (61) on Jc written as the product 
of the Jordan norms. 



Table 4.7 Non-Hermitian pairs (9, 6) with Omin H 9r 7^ 
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p,q > 5 throughout the table 

Comparing Tables 4.6 and 4.7, we find 

Proposition 4.8. There is a bijection between (i) the pairs (9,^) in Table 4.7 and (ii) the 
triples {J', P) where J is a Euclidean real Jordan algebra and P is a monomial in the Jordan 
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norms P[„] of the simple components of J such that each P[„] occurs at least once in P and 
P has total degree 4. 

Notice that the condition that J' has degree < 4 is necessary but not sufficient for J' to 
occur here (indeed J cannot be a Jordan algebra of rank 3) and that the same J can give rise 
to different polynomials P and hence different q (this occurs for = G2 and g = sog). 

Prom Table 4.7, we get in Table 4.9 a list of the real semisimple groups Gr occurring here. 
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In the listing of the exceptional real groups, the subscripted number in parentheses is equal 
to dimjK pR — dimR 6r and serves to distinguish between simply-connected real forms having 
the same complexified Lie algebra. In the first three cases d is the "correct" parameter for the 
corresponding simple Jordan algebra t_i while in the next five cases d is a fictitious parameter 
which we make up as it gives consistent formulas in Tables 4.9 and 6.9. 



5. Holomorphic Half- Form Bundles on Or. 



In this section, we construct and classify all holomorphic half- form bundles over Ojj 
equipped with its instanton Kaehler structure J from §2. Right away, we identify (Or, J) with 
the complex cone Y" in p by means of the Vergne diffeomorphism (2.3). 

We are assuming, throughout the rest of the paper, that Or is strongly minimal and (g, 6) 
is non-hermitian. Then 6 is a semisimple Lie algebra and p is irreducible as a representation of 
K. The spaces Or and Y are given by (3.7). In particular Y is the conical i^-orbit of highest 
weight vectors in p. The cases occurring here were classified in Table 4.7. 

Each holomorphic half-form bundle N2 over Y is automatically homogeneous under K 
(see Lemma 5.2). The space H = r(F, N2) of global algebraic holomorphic sections breaks 
up under the action of K into a multiplicity free ladder decomposition which we analyze in 
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Lemmas 5.2 and 5.3. 

In Proposition 5.5 we determine the spectrum of the operator E' on H given by the Lie 
derivative of the holomorphic Euler vector field E. We find that E' is diagonafizable with 
positive spectrum. This result about E' is crucial since it allows us to invert E' and + 1 in 
§6 in order to quantize the symbol g^o from Theorem 3.6. 

We regard E' as the "energy" operator on the space H of quantization. Indeed, E' = 
Q(A) = Q{p) by Corollary 2.2(i). I.e., E' is the quantization of our chosen Hamiltonian p (see 
discussion before (2.8)). Since p is positive everywhere on Or, the positivity of E' is exactly 
what we expect from the quantum theory. 

The Euler C*-action on Y (see §2) defines a complex algebra grading 

^(^) = © My) (5-1) 

where 

R^{Y) = {feR{Y)\Ef=pf} (5.2) 

Here denotes the set of non-negative integers. A priori, the grading in (5.1) extends over 
all integers, but since Y is the orbit of highest weight vectors in p, it follows that the pullback 
map ^(p*) R{y) on functions is surjective. So Rp{Y) = for p negative. 

Our first aim is to compute the fundamental group of Y. To do this, we use fact that Y 
is a homogeneous space of K: 

Y ~ K/K^ (5.3) 

Since Tri{K) = 0, it follows that Tri{Y) is isomorphic to the component group of K"^. 

Now let Q C K be the (closed) subgroup which preserves the line Ce. Clearly lies in 
Q as the kernel of the action of Q on Ce; we put Q' = K^. The quotient P(y) of Y by the 
Euler C*-action identifies with K/Q. Since Y is an orbit of highest weight vectors, ¥{Y) is a 
(generalized) fiag variety of K. So we have a i^-equivariant principal C*-bundle 

Y P(F) ~ K/Q (5.4) 

It follows easily that Q is the connected subgroup of K with Lie algebra q = to ® ^^i ■ Thus 
we get the Levi decomposition 

Q = Kq^Ki (5.5) 

where Ki = cxp^i is the connected unipotent subgroup of K with Lie algebra ti. 

Lemma 5.1. Y is simply- connected except if g is of type A^. If g = s[(ra, C), then the 
fundamental group ofY is 7ri{Y) c::: Z2 if n > A or tti{Y) ~ Z4 if n = 3. 

Proof. The discussion above gives K"^ = Kq tK Ki where Kg was defined in (4.3). The 
exponential map exp : ti ^ Ki is an isomorphism. Hence, the component groups of K*^ and 
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Kq identify. So 7ri(Y') is isomorphic to the component group of Kq = Kerx where x is the 
iiTo" weight of e. 

Proposition 4.3 says that x is the square root of the weight of P. (The square root is unique 
since Kq is connected.) Thus the product decomposition (4.13) gives = Xi^^ ■ ■ ■ xf"^ where 
Xn is the weight of the Jordan norm P[„] . So 

x=xr---xr (5-6) 

The well-known theory of the i^o-action on S{ti) (see [B-K3, Theorem 3.4 and Corollary 
3.6]) says that Xii--- iXe primitive characters of Kq and form a basis of the character group. 
The kernel of a primitive character is connected. It follows that the component group of Kerx 
is isomorphic to TLj gTL where g = gcd{t«i, .... we}. Now the Lemma follows immediately from 
Table 4.7 where P is given explicitly in the third column as a monomial in P[i], . . . , P^^ so that 
the exponents wi, . . . ,Wi can be read off in each case. ■ 

It is often convenient to label irreducible representations of K by their highest weight 
in the sense of the Cartan-Weyl highest weight theory. This requires that we fix a choice of 
Cartan subalgebra f) in 6 together with a Borel subalgebra b in 6. The set 7^"*" of non-zero 
weights of f) on b is the set of positive roots. Then TZ = TZ'^ U —7^"*" is the set of all non-zero 
weights of f) on g. Each a e IZ is called a root and the corresponding weight space P is 
1-dimensional and is called the a-root space. Then 

« = t)emem" (5.7) 

where m C b is the span of the positive root spaces and is the span of the negative root 
spaces. 

Cartan-Weyl highest weight theory says that in each finite-dimensional irreducible t- 
representation V, the subspace V"^ of all vectors annihilated by the action of m is 1- 
dimensional. Clearly then V"^ is a weight space of f) of some weight fi. Moreover V"^ turns 
out to be the full //-weight space in V. Then /x is the so-called highest weight in V, any 
(non-zero) vector in is called a highest weight vector, and we write V = V^. 

We choose (f), b) so that /i G f) and e is a highest weight vector in p. Wc also require 
that 6i lies in m, f) is complex conjugation stable and I) is stable under the complex Cartan 
involution of the symmetric pair (to,t')- It is easy to meet these conditions. Then 

pc^V^ (5.8) 

where ip is the weight of e so that V is the highest weight of p. Notice that is just the 
restriction to I) of the 6o-weight = dx ^ • 

Prom now on in this section, we work in the category of algebraic holomorphic complex 
line bundles L over a smooth (irreducible) complex algebraic manifold X, which will soon be 
specialized to X = Y. R{X) and T{X,L) denote the regular (i.e., algebraic holomorphic) 
functions on X and sections of L. 
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Next we discuss some general notions about half-form bundles. Let L — > X be a complex 
line bundle. A square root of L is a pair (C, a) where C is a complex line bundle over X and 
a : C®^ ^ L is a bundle isomorphism. Notice that a gives an R(X)-module map 

ax:r{X,C)^^^r{X,L) 

so that the product of two sections of C defines a section of L. 

Two square roots (C, a) and {C',a') are isomorphic if there exists a bundle isomorphism 
I3:C^C' such that a = a'o^®^ ^Yiere (3®'^ : C®^ ^ (C)®^ is the obvious map. In counting 
or classifying half- form bundles, we will always work up to isomorphism. 

It is easy to check that if s G T{X,L) is a non-zero section then there exists, up to 
isomorphism, at most one square root (C, a) of L such that s is the square of some section of 
C. In practice, we suppress the isomorphism a from the notation. Notice that any line bundle 
is a square root of its square. 

We will use the notation and terminology from [B-K3] on algebraic holomorphic differential 
operators and their symbols. If r/ G L), i.e., rj is an order 1 differential operator on 

sections of L, then r] determines an order 1 differential operator on sections of any square 
root C of L in the following way: there exists a unique operator 77" G V^{X,C) such that 
77(5^) = 2sri^{s) for all s G r(X, C). Then the symbols of 77 and 77'' coincide. We write 77 for 77" 
when the meaning is clear. 

The next result follows easily using general facts about homogeneous line bundles for the 
first part and [B-K3, Lem. 2.9 and Appendix §A.12] for the second part. In particular, the 
proof of the second part uses the Borel-Weil theorem on P(y) . 

Recall that /o G R{Y) was defined in (3.15). Using the terminology of this section we see 
that /o is a highest weight vector in R\iY) of weight t/^ and /o is Q-semi- invariant. 

Lemma 5.2. Suppose C is a square root of a K -homogeneous line bundle on Y . Then C has 
{uniquely) the structure of a K -homogeneous line bundle. The space H = T{Y, C) of global 
sections is non-zero. The differential of the corresponding K -action on H is a representation 
oft on H by differential operators of order 1 and is compatible with tensor product of bundles. 

The t-representation on H is completely reducible and multiplicity-free. A vector in H is 
a highest weight vector for the t-action if and only if it is Q -semi-invariant. The space H"^ 
of highest weight vectors has a basis of the form {/qSq \p G Z_|_} where sq & H is uniquely 
determined up to scaling. This gives a ladder decomposition 

^ ^ K+p^ (5.9) 

of H as a ^-representation where v is the "^-weight of Sq. 

We write if[^_|_p^] for the subspace of H which carries the ^-representation Vi^+p^. The 
representation V^, is the minimal ^-type in H. We may also call its carrier space the 
minimal t-type in H. 

A square root N2 of the canonical line bundle N = f\'^{T*X) on X, where n = dimcX, 
is called an algebraic holomorphic half-form line bundle on X. Sections of N2 are called half- 
forms. Let denote the Lie derivative of a vector field ^ on X so that in particular we get 
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an operator G ^^{X, N). The Lie derivative on half-forms is the operator G 'D^{X, N^) 
given by 

C^{s^) = 2sC^{s) (5.10) 

In this way we get a representation 

Vect{X) V{X, ) ^ End ^ ^ (5.11) 

where QJect(X) is the Lie algebra of algebraic holomorphic vector fields on X. Then on the 
symbol level 

symbol/:^ = symbol e R[^{T*X) (5.12) 

where R^j (T*X) denotes the space of regular functions on the cotangent bundle T*X which 
are homogeneous of degree p on the fibers of the natural projection T*X X. 
Now the Lie derivative of the Euler vector field E onY is the operator 

E' = Ce (5.13) 

The following result is easy to verify and defines the minimal £^'-eigenvalue tq . 

Lemma 5.3. Suppose Ni is a half-form bundle on Y . Then the K-action on gives a 
natural representation of K on the space H = r(y, N2) of global sections. Differentiating this 
gives the representation 

vr^ : « ^ I?(y, N* ) ^ End H, L^. (5.14) 

E' is diagonalizable on H with spectrum ro + Z+ where ro G This defines vq. Thus 
we have the eigenspace decomposition 

= Hr,+p (5.15) 

where Hq is the q-eigenspace of E' in H. The action of E' on H commutes with the K-action 
and the eigenspaces H^^^p are the irreducible K -submodules in H . Furthermore H^-g^p carries 
the representation Vi,-^-p^ so that 

Hro+p = -f^[i/+p^/>] (5.16) 

Let So G ilr-o be a non-zero Q-semi-invariant section. Then {f^so G Hr^+plp G Z+} is a 
complete set of linearly independent Q-semi-invariant sections in H. 

The Hilbert space of our quantization of Ok will be a certain completion of i? = r(F, N^). 
We think of E' as the energy operator and call Hrg the vacuum space in H. The sections in 
Hj-g are the vacuum vectors in H. The vector sq chosen in Hj-g is unique up to scaling. 
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A spherical -fC-representation is one that contains a non-zero i^-invariant vector, which is 
then cahed the spherical vector. It follows from Lemma 5.3 that H is i^T-spherical if and only 
if — C, so if and only if .so is iC-invariant. 

Remark 5.4. Our discussion of square root bundles and half-form bundles generalizes in the 
obvious way to nth root bundles and nth roots of the canonical bundle. Then all the results in 
Lemmas 5.2 and 5.3 go over to the case where we replace n4 by an nth root of the canonical 
bundle, the only change being that then ro G instead of Vq G ^Z. 

Our key result on half-form bundles is 

Proposition 5.5. Suppose Ni is a half-form bundle on Y. Then the minimal eigenvalue Tq 
of E' on H = r{Y,N2) is positive. Thus E' has positive spectrum ro -I- Z_|_ on H. 

For the proof, we need to construct a concrete holomorphic (m -|- l)-form A on Y . We will 
use this form A throughout §5 and §7.. We construct A out of the functions fo, fi ■ ■ ■ , fm on 
Y we defined in (3.15) in the following way: 

A = d/o A d/i A • • • A d/^ G T{Y, N) (5.17) 

Since /o, /i . . . , fm were coordinates on the open set Y° defined in (3.17), it follows that A is 
nowhere vanishing on Y". Furthermore A is Q-semi-invariant and 

C = Q- weight of A (5.18) 

is the character by which Q acts on p2 ® A™pi. 

Proof of Proposition 5.5. We will show that the spectrum of Ce on r(Y', N) is positive. 

This implies the positivity of the spectrum of E' on H. 

It follows by Lemma 5.2 that r(K, N) is a multiplicity-free A'-module of ladder type and 
the set {fP~^A\p G Z+} is a complete set of linearly independent highest weight vectors, 
where /? is some non-negative integer. Let 

Ao = fo^A (5.19) 

Since E(fy) = fy for all u G p and C^h[fy) = kfy if u G pk, we find 

jC-e(Ao) =tAo with t = -d + l + m 

(5.20) 

C^h{Ao)=jAo with j = -2/3 + 2 + m 

But j > 0. This follows from the representation theory of the Lie algebra I defined in (4.7) as 
Cri'^{Ao) = for all x e ti and so in particular for x = e^. Therefore Aq is the highest weight 
vector of a finite-dimensional irreducible l-representation. So 2t = j + m is positive and hence 
t is positive. This gives the result as i -|- Z_|_ is the spectrum of E' on T{Y, N). ■ 

Proposition 5.6. 
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(i) If g is not of type An- then up to isomorphism Y admits at most one half-form bundle. 

(ii) If Q = sl(n, C) (n > 3), then either Y admits no half-form bundle or Y admits exactly two 
non-isomorphic half-form bundles. 

(iii) //N^ is a half-form bundle on Y with H = r(y, N^) and so G is a non-zero Q-semi- 
invariant vector, then up to scaling 

either Sq = Aq or Sq = /qAq (5-21) 

Moreover the two possibilities in (5.21) classify half-form bundles on Y up to isomorphism. 

Proof. Prom formal properties of square root bundles, it follows that if the canonical bundle 
N on y admits any square root, then the set of all square roots up to isomorphism is pa- 
rameterized by the order 2 characters of tti(Y). Thus by Lemma 5.1, if y admits a half-form 
bundle then it is unique with the exception of the cases g = sl(n, C) where there would be two 
half-form bundles. This proves (i) and (ii). 

Now suppose we are given N2 and so- Recall Aq from (5.19). The ladder structure on 
r(y, N) (Lemma 5.2) implies that (up to scaling) Sq = /qAq where p G Z_|_. If p > 2 then we 
consider the local section f^^so € r(y°,N2). The square of fQ^So is equal to /q~ Aq and 
this has no poles on Y. It follows that fo^so has no poles on Y and so fo^so G r(y, N2). 
But fo^so has £^'-degree equal to ro — 1 and so this contradicts the minimality of tq. Thus 
p = or p = 1. Furthermore, Aq and /qAq cannot both be squares of sections of the same 
bundle N2 since, by (5.1), /o G -Ri(y) is not a square in R{Y). ■ 



Proposition 5.7. 

(i) // = SO{p, q) where p + q is odd and p,q > 4, then Y admits no half-form bundle. 

(ii) If Gm. = SL{p,R) where p is odd and 5 < p, then Y admits no half-form bundle. 

— i 

(iii) If Gm_ = SL{n,M.) where n> A is even, then Y admits exactly two half-form bundles N|. 

1 

and . These may be characterized by the conditions: Aq is the square of a section of 

and /qAq is the square of a section of Nf. . 

(iv) In all other cases, Y admits a unique half-form bundle N2 and Aq is the square of a 
section ofN^. 

All the half-form bundles occurring here are listed in Table 6.9 along with their minimal E' - 
eigenvalue ro and the t-type V^, of the vacuum space Hj.^. 

Proof. By (5.19) the form Aq is i^'o-semi-invariant of weight Co = X~^C- It follows easily 
that y admits a half-form bundle with Sq = Aq (respectively Sq = /qAq) if and only if 
(respectively xCo) is the square of a character r of Kq. Then r is unique and dr = u is the 
highest weight of if^o ■ 

Thus we need to compute the character and look for square roots in the character group 
of Kq. The Jordan structure on 6_i gives us a convenient way to do the calculations. Indeed 
we found in the proof of Lemma 5.1 (i) the basis XIt ■ ■ iXe of the character group of Kq and 
(ii) the formula (5.6). 
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Now by (5.17) A transforms in the 1-dimensional J^o-representation 

P2®A>i ~ x"'^^®A"^«_i (5.22) 

But A'"^-! is the tensor product of the top exterior powers of the spaces . . . , The 
weight of Ko on the top exterior power of is where Un = 2 + dn{qn ~ !)■ Here d„ is 

the root multiphcity parameter of given in Tabic 4.6 and the formula for u„ is immediate 
from the description of the restricted root system of (to)^')- Thus the weight of A is by (5.6) 

^ — X Ai ' "Xi — Xi ' "Xe, yo.zn) 

We can decide if a section s = /q A over Y° extends to Y just by examining its weight 

Xi^ ■ ■ ■ xl'^ ■ Indeed s extends to Y if and only if ti, . . . ,ti > 0. This follows easily by using 
the Borcl-Wcil theorem on ¥{Y) (in its geometric form involving the orders of poles along 
irreducible divisors in the complement of the big cell) to compute r(y, N). Thus, going back 
to the definition of Aq = fo^^ in (5.19), we see that /3 is the largest non-negative integer 
such that all the numbers —(3wn + (m + l)wn — Un are non-negative. To simpliiy, we put 
a = m + 1 — (5. Then the weight of Aq is 

- Xi ■ --Xl lt».24j 

where a is the smallest positive integer such that aw^ > Un for all n. 

It is now easy to go through Table 4.7, calculate Co in each case, and see if Co and/or 
xCo admits a square root. The cases group together naturally into families. In the first three 
cases in Table 4.7 we have P = -P4,r, i-*4,C) -P4,h so that £ = 1, wi = 1, ui = 2 + 3d. Then 
a = 2 + 3d and Co = 1- Thus we get a half-form bundle (unique as g 7^ sl{p, C)) with Sq = Aq 
and Hj.^ ~ C. 

In the four cases where P = P^^rPi, we have i = 2, {vji,W2) = (1,1) and (1*1,^2) = 
(2 + 2d, 2) where d = di. Then a = 2 + 2d and Co = x¥ which is a square. So we get a unique 
half- form bundle with Sq = Aq and H^g ~ C (8) S'^C'^. The G2 case P = PfP[ is similar with 
{'W\,W2) = (3, 1) and (iti,«2) = (1, !)• Then a = 2 and Co = Xi which is a square. This gives 
a unique half-form bundle with Sq = Aq and we find iJ^o — 'S'^C^ (g) C since x = X1X2 is the 
highest weight of p = S^C^ C^. 

We can treat all the cases where = 5o{p,q), 3 < p < q, simultaneously with ^ = 2 as 
long as we formally set J(2;4) = J{1) J(l), P2;4 = PiPi, and J(2;3) = J(l), P2;3 = Pi 
with S'^/^C^ = S'^C^ as so (3)-representations. This follows easily and we get {wi,W2) = (1, 1) 
and {ui,U2) = {p — 2,q — 2). Then a = q — 2 and Co = Xi~^- If (? — f is even, then Co 
is a square, and we get a half-form bundle with Sq = Aq and Hr„ 5'('J-p)/2([^2 ^ since 
X = X1X2 is the highest weight of p ~ ® C'^. This is the unique half-form bundle unless 
{P: q) = (3, 3) in which case we get a second half-form bundle with Sq = /qAq and -ff^^ — p 
(where A denotes Cartan product) so that iJ^o — (g) C^. Indeed so(3,3) = sl(4,R) and 
this is the only time when our so{p,q) cases and s[(n, M) cases coincide. Now if g — p is odd 
with p> 3, then neither Co nor xCo = (Xi~^^^) X2) are squares. However, if g — p is odd with 



31 



p = 3, then Co? but not xCo, is a square. Thus we get one half-form bundle with Sq = Aq and 

Hr, ~ 5('?-3)/2(C3) C = S"?-3C2 C. 

Finally we consider the = sl{p,R) cases. If p > 5, then P = Plipi ^ = 1, = 2, 
■ui = p — 2. So if ^) is even then a = {p — 2)/2, (o is trivial and we get a half-form bundle 
with Sq = Aq and iJ^o — C. The second half-form bundle has Sq = /oAq and H^"^ ^ p so that 
Hj-Q c± C^'. If p is odd, then there is no half-form bundle. We already did the case p = A. If 
p = 3 then P = Pf, £ = 1, = 4, ui = 2 and so a = 1 and = Xi- We get one half-form 
bundle with Sq = Aq and ^ C^, and a second with Sq = /qAq and Hrg ^ 5^C^. 

We have now proven everything except for the values of Tq. But (5.19) and (5.20) give 
E'Aq = (— /? + m + l)Ao = ckAq. So if Sq = Aq then vq = a/2 while if Sq = /qAq then 
ro = (« + 1)/2. We have computed the parameter a in each case above, and this produces the 
values of ro in Table 6.9. ■ 

Remark 5.8. Case (i) is the Howe-Vogan counterexample. Howe proved that these groups 
SO{p,q) admit no minimal unitary representation and then Vogan ([Vol]) extended this to 
the simply-connected covering groups. 

The isomorphism (3.16) implies in particular that R{Y°) is the localization of R{Y) at /o 
so that 

R{Y") = R{Ym'] = C[/o, /i, . . . , fm][fo'] (5.25) 

It is easy to prove 

Proposition 5.9. Suppose C is a K -homogeneous line bundle on Y and s G T{Y,C) is a 
non-zero Q-semi-invariant vector. Then s is nowhere vanishing on Y°. Consequently, since 
Y" is affine, the space of sections T{Y",C) is a cyclic R{Y°)-module generated by s so that 

r(y°,C) = T{Y,C)[f^'] = R{Y'')s (5.26) 

In particular then, in Proposition 5.6, H = T{Y°, N^) is a cyclic R{Y°)-module generated 
by any section /qSo- 

This is a key result as it enables us to analyze H ina uniform manner in §7 below regardless 
of whether H is i^-spherical or not. In fact, we further simplify our work in §7 by making the 
following observation, which obviates the need to consider separately the two possibilities in 
(5.21). 

The regular function /o G Ri(Y) is not a square in R{Y°) (because of (5.1) and (5.25)) 
and is nowhere vanishing on Y° by the definition of Y°. Thus we may construct a non-trivial 
two-fold etale covering 

y'" ^Y" (5.27) 

by "extracting a square root of /o". Then Y°, like Y°, is again a smooth affine complex 
algebraic variety and has a unique i^o-action such that the cover Y° Y° is i^o-equivariant. 
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The pull back of N 2 through the covering is a i^Co-homogeneous bundle on Y" which we again 
call N2. Now Proposition 5.9 gives 

Corollary 5.10. The space r(F°,N^) of algebraic holomorphic sections is a cyclic R{Y°)- 
module generated by VA so that 

T{Yo, N^) = C[/o* ,f-\h,..., fm]VA (5.28) 



§6. Quantization of Or. 

In this section we construct explicitly our quantization of Ok- This is purely "from 
scratch"; we assume no a priori information on the existence of any quantizations or unitary 
representations. In the next section we prove that the constructions of this section "work", 
i.e., we prove Theorem 6.3 and 6.8. 

The first step of our quantization, carried out in §2 and §3, was to transform the quantiza- 
tion problem on Or into a quantization problem on T*Y. We replaced (by holomorphic exten- 
sion) each function cj)^ , w G Qk, by a rational (pseudo-differential) symbol G R{T*Y)[X~^]. 
Then, after complexification, we ended up in (3.6) with a realization of g as a complex Lie 
algebra of rational symbols, 

e R{T*Y)[X-% zeg (6.1) 

Our aim now is to quantize the symbols z € g, into operators Q(<I>^) on a Hilbert 
space TC which is a completion oi H = r(y, N2) for some half-form bundle N2 over Y. As 
usual, we freely identify (Or, J) with Y via the Vergne diffeomorphism (2.3). We require our 
operators satisfy certain explicit and implicit axioms. This solves our quantization problem 
on Or as we set Q((/)«') = Q($^), w e 0r. 

In §5 we have already quantized the symbol A, corresponding to our chosen Hamiltonian 
p (cf. Corollary 2.2(i)), into the operator E' on half-forms. 

We require that the operators Q($"'), w G gR, be self-adjoint, or equivalently, that the 
operators Q($^) satisfy 

Q($-)t = Q($^), zEQ (6.2) 

Of course at this point, H carries no preferred positive definite Hermitian inner product. So 
we will have to construct this along the way. Our operators will not be defined everywhere on 
H, but they will all contain H in their domain. 

We further require that the Dirac commutation relations (2.4) be satisfied. If we put 

TT- = iQ($-), zEQ (6.3) 
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then these relations amount to 

[vr^TT^'] = 7r[^'^'l (6.4) 

for all z^z' G fl. I.e., the map tt : g ^ Endi?, z tt^, must be a complex Lie algebra 
homomorphism. 

In order to satisfy the implicit axioms, we require that the "symbol" of Q(<1>^) is just 
Here our definition of "symbol" is not precise, as we found in Theorem 3.1(iii) that the symbols 

w G p are not homogeneous. However, we will get around this by dealing individually with 
the homogeneous pieces. 

With this in mind, we mandate 

TT^ = iQ($^) =/:^., x^l (6.5) 

So TT^ is the Lie derivative on half-forms of the algebraic holomorphic vector field if defined 
in (2.6) by differentiating the i^-action on Y . Thus, just as we would expect, tt^ corresponds 
to the natural iC-action on i.e., vr^ = •7r^(x) in the notation of (5.14). 

Next, guided by the complex Cartan decomposition (3.5), we need to quantize the symbols 
11 G p. In (3.4) we found these break into a sum /„ + of two homogeneous pieces. We 
now mandate 

T,- = iQ($-) = if^ + ir„ vep (6.6) 

where = Q{gv) is some "nice" quantization of the homogeneous degree 2 rational symbol 
from (3.4). 

This leaves the problem of how to construct Ty. Of course we want in the end for vr to be 
a complex Lie algebra homomorphism. So we certainly want [7r'^,7r''] = tt'^'"] and this implies 

[7r^^„]=^[,,,] (6.7) 

Hence the fact that p is irreducible as a ^-representation insures that T^^ and the tt^, a; G ^, 
already determine all operators T^, G p. 

To construct T^^^ we break down the symbol g^, = g^^ computed in Theorem 3.6 by (3.25) 
into its elementary factors. On the face of it, it seems hard to imagine what to do with the 
factor f^^i^xP) appearing in (3.4). While is the symbol of the perfectly nice order 

4 differential operator 7r^(P) on sections of N2 because of (4.5) and (5.14), the quotient 
fo^i^xP) a priori only defines a differential operator on sections of the bundle N2 restricted 
to the open set Y" from (3.17). Fortunately, the result in [B-K3, Ths. 3.10 and 4.5] (which 
applies more generally to any homogeneous line bundle over Y) tells us 

Theorem 6.1. Suppose is a half-form bundle on Y and H = r{Y,N^). Then the 
operators /o and tt^P on H commute and have the same image. Hence the formula 

De = U'^kP) (6.8) 
Jo 
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defines an operator on H. It follows that is an algebraic differential operator of order 4 on 
sections 0/ N2 . 

The assignment e ^ extends naturally and uniquely to a complex linear 1-to-l K- 
equivariant map 

p^V{Y,N^), v^D^ (6.9) 

ofp into the algebra P(y, N^) of algebraic differential operators on sections o/N^. Then, for 
each non-zero v e p, has order 4; also has degree —1, i.e., [E' , D^] = —Dy. 

The subalgebra A C N2) generated by the Dy, v £p, is abelian, isom,orphic to R{Y) 
and graded by A = ®p>oA-p where A-p = {D ^ A\[E' , D] = —pD}. Putting Dy = Df^ for 
V & p, we get a graded K-equivariant complex algebra isomorphism 



R{Y)^A, f^Df (6.10) 

There is a unique K^-invariant positive-definite Hermitian inner product on H such 
that Bo{sq,sq) = 1 {when we fix a choice of sq in Lemma 5.3) and the operators fy and 
Dy are adjoint with respect to Eg for all w G p. Then the grading (5.15) is a Bg-orthogonal 
decomposition. 

The expression for in terms of our local coordinates (3.15) on Y is, in the notation of 
(3.24), 

D, = flP{C^,...,Ce_) (6.11) 

To complete our quantization of the rational symbol g,,, we need to quantize the factor 
— in (3.25). It is natural to try to quantize — into the operator (£" + a)~'^{E' + b)~^ 
where a and b are some constants to be determined. Of course a and b must be chosen so that 
neither —a nor —b belongs to the spectrum of E' . 

In fact it turns out that exactly one choice, namely a = and 6 = 1, satisfies our require- 
ment that the resulting operators defined by (6.5) and (6.6) satisfy the bracket relations of g. 
In fact, just the one relation [tt*^, tt'^] = tt'* mandates that 

quantizes to -^r^^r^^ (6-12) 

(We prove in §7 that this choice works.) We emphasize that the operators E' and E' are 
invertible since the spectrum of E' on H is positive by Proposition 5.5. 

Thus, rather than putting Ty = (^E'+a){E'-\-h) ^'^'^ solving for a and b in §7 we simply 
define 

In [B2, proof of Theorem 4.2], we wrote out the latter procedure of determining a and b from 
the bracket relation for the case 0m = s[(3,]R). 
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These operators are no longer differential operators, but they share many of the same 
properties. To explain this, we introduce the notion of fi-finite endomorphism. 

We have a natural representation of 6 on End if defined hy x * D = [>C^a!,£)]. Then 
D G EndH is called i-finite if D generates a finite-dimensional representation of i inside 
End H. The space End i-finH of all 6-finite endomorphisms of iJ is a complex subalgebra of 
Endil. Let 

End \p]H d End i-finH (6.14) 

be the p-eigenspace of &dE' . We have a natural graded iT-equivariant complex algebra inclu- 
sion (see [B-K4, A.6, A.12]) V{Y,^^) C Ende_/i„ff. 

Now Theorem 6.1 easily gives (cf. proof of [B-K4, Th. 3.4]) 

Corollary 6.2. The operator Ty, v G lies End[_i]i?. Thus we get a K-equivariant complex 
linear map 

T : p ^ End v T^ (6.15) 

The pseudo-differential operators Ty, v e p, commute and generate a graded abelian K-stable 
subalgebra T = ®p>o'^-p of End ^-finH where T^p = T D End[_p]i?. We then get a graded 
K-equivariant complex algebra isomorphism 

R{Y) ^ r, f^Tf (6.16) 

where Tf^ = Ty for f G p. There is a K^-invariant positive- definite Hermitian inner product B 
on H , such that B{so, sq) = 1 (when we fix a choice of Sq in Lemma 5.3) and the operators fy 
and Ty are adjoint with respect to B for all v ^p. Then the grading (5.15) is a B-orthogonal 
decomposition. 

Taking inventory of our operators, we see that vr^, x G t, and fy, Ty, v G p are each graded 
operators on H of degrees 0,1, and —1 respectively. I.e., we have 

TT^-.Ht^Ht, fy-.Ht^Ht+i, Ty:Ht^Ht_, (6.17) 

Now we can state 

1 1 

Theorem 6.3. Suppose N2 is a half-form bundle on Y and H = r(Y', N2). Let 

vr : g ^ End t-finH, z ^ it'' (6.18) 
be the complex linear map defined by (6.5), (6.6), (6.13) so that 



TT^ = iQ($^) = ifxel, 
T,- =iQ{<t^)= if^+iTy ifvep 



(6.19) 
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Then, except in the one case where Gm = S'L(3,M), ro = 1, and Hi ~ the map TT IS 
a complex Lie algebra homomorphism so that tt is a representation of g by global algebraic 
pseudo-differential operators on sections 0/N2. 

Proof. As in [B-K4,§6], the problem reduces to proving the single bracket relation of operators 
on H 

[vr^ TT^] = tt'^ (6.20) 
because of [B-K4, Lem. 3.6]. Since the operators satisfy 

[fv,fv'] = m,n']=0 (6.21) 
for all v,v' G p, we get [vr^jTr'^] = [/e, T^e] — [fe,Te]. Thus (6.20) amounts to the relation 

[/e,re]-[/e,Te]=£^. (6.22) 

We prove (6.22) in §7. We also show that in the S'L(3,R) case we omitted, tt fails to be a Lie 
algebra homomorphism. ■ 

For the rest of this section, we assume that we are in the situation of Theorem 6.3 with 
the one bad case excluded so that tt is a Lie algebra homomorphism. Let 

n : U id) ^ End i-finH (6.23) 

be the complex algebra homomorphism defined by tt. Let £ be the image of tt and let X C ^ (g) 
be the kernel of tt. Then we have a natural complex algebra isomorphism 

U{5)/I ^ £ (6.24) 
Let (g) be the pth Cartan power of the adjoint representation of g. 

Theorem 6.4. The representation ir of g on H is irreducible. The algebra homomorphism tt 

is surjective so that 

£ = End i-finH (6.25) 

The algebra £ has no zero-divisors. Thus the annihilatorl of it is a completely prime primitive 
ideal in U (g) . 

We have £'p/£i'~^ c± ^'[^'(g) as g-modules and so there is a multiplicity free g-module 
decomposition 

£ ®pez+S^Kd) (6.26) 
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The associated graded ideal grX C S{q) is the prime ideal defining the closure of Omin- Thus 
the associated graded map to tt gives a graded algebra isomorphism 

i?(Omin)^gr^ (6.27) 



Proof. The proofs in [B-K4,§5] of the corresponding results go through verbatim (by design) 
into this more general setting. The only change needed is that we replace the line "Let 
So = 1 e Hj-Q^ in [B-K4] by "Let sq € -f^ro be a non-zero highest weight vector for the 
t-action." ■ 

If g 7^ s[(n, C), then Joseph ([J2]) proved that Z^(g) contains a unique completely prime 
primitive ideal with associated nilpotent orbit Omin (cf. §4). This is called the Joseph ideal. 
Thus we get 

Corollary 6.5. If Q sl{n,C) thenX is the Joseph ideal. 

We remark that this gives a new proof of Garfinkle's ([G]) result that the associated graded 
ideal of the Joseph ideal is prime. 

We now fix a non-zero vacuum vector sq G Hj.^ which is Q-semi-invariant, or equivalently, 
a highest weight vector for the ^-action. 

Theorem 6.6. H admits a unique Q^-invariant positive- definite Hermitian inner product 
{s\s') such that (so|so) = 1. This coincides with the inner product B found in Corollary 6.2 
so that 

B{s,s') = {s\s') (6.28) 

Consequently, the representation of gR on H given by n integrates uniquely to give a 
unitary representation 

TTo : Gm ^ Unit n (6.29) 

on the Hilbert space H obtained by completing H with respect to B. Then H is the space of 
K^- finite vectors in H. 

Proof. This follows by the proof of [B-K4, Th. 5.2], using the same modification described 
in Theorem 6.4. See, e.g., [W, §6. A. 4] for a proof of Harish-Chandra's theorem that the 
giR-action on an admissible finitely generated (g, i^)-module S endowed with a gM-invariant 
positive-definite Hermitian inner product integrates to a unitary representation of on the 
Hilbert space completion of S. We apply this with H = S. Indeed H is irreducible by Theorem 
6.4 and so generated by any non-zero vector. Also H is admissible (i.e., all ivT-multiplicities 
are finite) since H is in fact multiplicity free by Lemma 5.3. I 

We will write {s\s) = \\s\f. Theorems 6.4 and 6.6 give, in the language of §4, the repre- 
sentation theoretic result 
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Corollary 6.7. tTo is a minimal unitary representation of and H is its associated Harish- 
Chandra {q, K) -module. 

Theorem 6.8. There exist positive real numbers a and b {depending on Gu and N^) such 
that 



2 



{a)n{b), 



(6.30) 



n\{ro + l)n 

for all n G Z_|_ . Moreover a and b are unique up to ordering and satisfy 

a + 6 = ro + 1 + Xo (6.31) 

where Xq is the eigenvalue of C^^h on sq- We compute a and b below in Table 6.9. 

The values 11/(5*5011^ ctn-d K^-invariance uniquely determine the inner product B on H 
because of the ladder decomposition (5.15). 

Here we arc using the hypergeometric function notation (a)„ = a{a + 1) • • • (a + n — 1). 

Proof. The adjoint of multiphcation by /o = fe is T^ by Corollary 6.2. We find T-^f^so) = 
lkfo~^SQ for k G where 7fe is a scalar and 70 = 0. This follows by £^'-dcgree and weight as 
in [B-K4, proof of Th. 5.2]. Indeed, Te(/oSo) lies in Hrg+k-i and has (^-weight u + (k — 
But by Lemma 5.3, /o ~^So is a highest weight vector of weight i' + {k— in the irreducible 
^-representation iJ^o+fe-i- Thus Tg{fQSo) is a multiple of /q ~^so- We now find 

||/o"so||' = {so I T-"(/o"5o)) = 71 • • • 7n{so I so) (6.32) 

We evaluate the RHS of (6.32) in §7 below and obtain (6.30) and (6.31). The final state- 
ment that these values determine B is immediate from Lemma 5.3 — in particular the ladder 
decomposition of H is multiplicity-free. ■ 

Table 6.9. 



Case 


Gr 


^2 
*0 




ro 




a 




6 




(i) 


-E6(6) 


Ao 


C 




l + |d 


5 
2 


l + ^d = 


3 
2 


l + (i = 


2 


(ii) 


^7(7) 


Ao 


C 




l + fd 


= 4 


l + ^d = 


2 


l + d = 


3 


(iii) 


^8(8) 


Ao 


c 




1 + §d 


= 7 


l + id = 


3 


l + d = 


5 


(iv) 


^4(4) 


Ao 


C(8) 




l + d = 


2 


l + ld = 


3 
2 


l + d = 


2 


(v) 


-£^6(2) 


Ao 






l + d = 


3 


l + ^d = 


2 


l + d = 


3 


(vi) 


^7(-5) 


Ao 


C(g) 




l + d = 


5 


l + ld = 


3 


l + (i = 


5 


Vii) 


-£'8(-24) 


Ao 


C(8) 




l + d = 


9 


l + |d = 


5 


l + d = 


9 


Viii) 


^2(2) 


Ao 






1 




l + ld = 


4 
3 


l + d = 


5 
3 


(ix) 


Sd{p,q) 


Ao 




-P^/^CP (g) 


2 




q-2 
2 




q-p+2 
2 





3<p<g, p-l-q'is even 



39 



(x) 30(3, q) Ao S^-^C^^C 
4< q, q is even 



3r"2 ^ IT anl 21-2 q-1 

2 2 2 



xi) SO{3,3) /oAo C2(g)C2 1 

xii) SO{p, q) none * * 

4<p<g, p + g^is odd 



2 ^ ^"2 1 3 3 

2 2 



w-2 1 
4 2 



n 3 n+2 

4 2 4 



xiii) 5L(n,M) Ao C 

4 < n, n is even 

xiv) 5L(n,M) /oAo C" 

4 < n, n is even 

xv) )S'L(n,M) none * * * * 

5 < n, n is odd 

xvi) 5L(3,M) Ao C2 1 f f 

xvii) 5L(3,R) /oAo ^^C^ 1 * * 



In Table 6.9, the symbol * means that there is no entry because some aspect of the 
construction has failed. In Cases (xii) and (xv) there is no half-form bundle, while in Case 
(xvii) the operators tt^ fail to satisfy the bracket relations of s[(3,M). 

The final result we present here about our minimal representations is the computation of 
a matrix coefScient on the one parameter subgroup generated by x = e + e G pR. Indeed, the 
same arguments used in [B-K4, Th. 6.6] go though in this setting to give 

Theorem 6.10. We have, for t e M, 

((exp tx) ■ so\sq) = 2Fi{a,h;l +ro\ — sm}o? t) (6.33) 



7. Differential Operators on Half-forms and the Generalized Capelli Identity. 



The purpose of this section is to complete the proofs of the results in §6, i.e., to prove 
Theorems 6.3 and 6.8. We already reduced Theorem 6.3 to the operator relation (6.22) on H. 

To begin the proof of (6.22), we observe that the two operators [/e, — {ff,-,T£\ and C^h 
appearing on the LHS and the RHS of (6.22) are both i^^o-invariant. The first idea of the proof 
is to exploit this observation. Since Kq is reductive, it follows that H is completely reducible as 
i^Q-representation. So we can fix a direct sum decomposition H = ®aHa where each subspace 
Ha carries an irreducible i^o-representation. Now to prove the operator relation (6.22) holds 
on it suffices to prove that (6.22) holds on just one non-zero section in Ha- 

There is a natural method to pick a section from iJ^, unique up to scaling. This uses 
structure of Ha as a ^Q-representation. The method is to pick to be a lowest weight vector 
for ^Q. Here we appeal again to the Cartan-Weyl theory recalled in §5 (where we applied it 
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to H considered as a ^-representation) . This time we use lowest weights rather than highest 
weights just for convenience. 

To get the notion of lowest weight for Iq, we take the triangular decomposition 

«o = f)©moemo (7.1) 

induced by (5.7). Then f) is a Cartan subalgebra of to and bo = f) ® Ttio is a Borel subalgebra. 
So now a lowest weight vector in for to is a vector in the 1-dimensional space if„ ° . Then 

is the space of (i.e., spanned by) all lowest weight vectors in H for to- 

So proving (6.22) reduces to verifying it on each lowest weig ht vector s° G ■ By 
iiTo-invariance and Schur's Lemma , the vectors ([/e, Te] — [/e, Te])(s") and C^h{s°') again lie 

in Ha° ■ Consequently, for s = s° we have 

jC^h{s) =Xs 

([/e,re]-[/e,re])(s) =Rs 

where X and R arc scalars depending on a. So proving (6.22) reduces to showing that, for 
each a, the scalars X and R coincide. 

The second, more serious, idea is to work out the first idea using the Jordan algebra 
structure on t_i from §4. It turns out that the Jordan structure gives us (i) a nice way to 
write down a basis of H"'^n and (ii) a means to compute R in (7.3) in the form of the generalized 
Capelli Identity of Kostant and Sahi ([K-S]). The computation of X follows easily from (i). 
Then, with everything computed, we see manifestly that R = X. 

The rest of this section is devoted to working this out. To start off, we embed iJ in a 
larger space il" which is easier to work with. We choose the natural embedding 

H C = T{Y°,N^) (7.4) 

We constructed the 2-fold covering Y° of Y° in (5.27) and then the pullback bundle (again 
called) N2. Clearly H sits inside i?" as the space of sections which descend to Y° (i.e., are 

(Z/2Z) -invariant) and then extend to all of Y. 

In Corollary 5.10 we got an nice description in (5.28) of HK Since /o and \/A are Kq- 
semi- invariant, we might as well rewrite (5.28) as 

JT«= (c[/i,...,/J®C[/o^/o"^]) VA (7.5) 

This makes it clear that a decomposition of the polynomial algebra C[/i, . . . into irre- 
ducible i^o-representations will produce a decomposition i?" = (BaH^ into irreducible Kq- 
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(7.3) 



representations. In particular we have 

(H^o = (c[/i,...,/„rc ®C[/o*,/o"*]) VA (7.6) 

We will deal with the problem of locating H"^o inside (H^)"^" when the time comes. 

Now we can bring in the Jordan algebra We have a vector space isomorphism 6_i 
Pi: y '-^ [Ui^]': cf. Lemma 3.4. This induces a graded complex algebra isomorphism 

5(e_i)^C[/i,...,/J, g^g (7.7) 

defined in degree 1 by y = f[y,e] ^oi y G t_i where fy was defined in (3.1). This isomorphism 
has weight degree p under the action of Kq. Hence (7.7) is i^o'^Q^i'^^^i^^^ ^^'^ gives 
by restriction a graded complex algebra isomorphism 

5(«_l)-0 ^C[/l,...,/^]-0, g^g (7.8) 

Recall from (4.12) that g„ is the degree of the Jordan algebra )[„]. We put ci = and 

Cn = qi-\ ^Qn-i, n = 2,...,£ (7.9) 

We have the well-known result (see [B-K3, Theorem 3.4 and Corollary 3.6]) 

Lemma 7.1. The natural representation of Kq on S{t-i) is completely reducible and multi- 
plicity free. 

The ring of lowest weight vectors in S{i-i) for the Ko-action is a polynomial algebra in q 
independent graded generators so that 

5(«_i)"^o =C[iVi,...,iV,] (7.10) 

The polynomials Ni,. .. ,Nq are uniquely determined by the conditions (i) for I < j < qn, we 
have N^^+j e 5'J'0[n]) and (ii) Ni{ei) = ■■■ = Nq{et) = 1. 

Now combining the work we have done thus far in this section we obtain 

Proposition 7.2. The ring of lowest weight vectors in C[/i, . . . , fm] for the K^-action is a 
polynomial algebra in the q independent graded generators Ni,. . . ,Nq so that 

C[/i,...,/„ro =C[7Vi,...,7Vj (7.11) 

Then degN^r^+j = j for 1 < j < g„. 

Suppose N2 is a half-form bundle on Y and H = r(y, N2). Then the natural Ko-action 
on H is completely reducible and has a basis of lowest weight vectors of the form 

s = f^Nl' ■■■Nl''VX (7.12) 
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where p & and ti, . . . ,tq G Z+. Here we regard s as a section in the larger space HK The 
section s in (7.12) determines p,ti, . . . ,tq uniquely. 

The KQ-representation on each E' -eigenspace Hr^+n is multiplicity free. 

If we range over all tuples p,ti, . . . ,tq in (7.12), then we obtain a basis of {H^)'^o . We 
could explain how to decide when the corresponding section lies in H, but we omit this as it 
is not necessary in our work below. (Only the partial answer we give later is needed.) 

Now that we have obtained a nice basis of H^o as promised, we need to start computing 
the eigenvalues of our various operators on the section s in (7.12). We set t = (ti, . . . , tq). We 
put 

g 

= 7V*i . . . iV*« and ^ = deg iV* = ^ ti deg A^^ (7.13) 

i=l 

Recall from (3.14) that m = dime Y — I. 

Lemma 7.3. Suppose s & H is of the form (7.12) Then 

m + 1 



E's =rs where r = p + z + ^ 

m + 2 

Cjjh s =Xs where X = 2p + z -\ — 



(7.14) 



Proof. We have 

E'{s) = iEf^N')VA + fSN^iE'VA) = (p + z)s + ^^s (7.15) 

using (7.12) and the formulas (3.19) and (5.17) for E and A in terms of our local coordinates. 
Next 

Then computing the same way as in (7.15) we find 

Cr,n{s) = ir,^f^N')VA + f^N\C^uVA) = {2p + z)s+'^s (7.17) 



Now we have come to the tricky part, computing the eigenvalue R of [/e, Tg] — [/ej^V] on 
each section s from (7.12). Expanding out we get 

[/e') Tg\ [fei = fe'^e '^efe fe^e "I" Tgfg 

- I f-D D f- fD-+ D-f ^'-'^^ 

{E' - 1)E' " E'{E' + 1) {E' - 1)E' " E'{E' + 1) 
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We will use the generalized Capelli Identity of Kostant and Sahi ([K-S]) to compute the 
eigenvalues of f-De, Dgfe, feDe, D-f^, , and on s. The idea is to transform this computation 
into a computation on ^(^-i). This works because we will write everything in terms of our 
local coordinates /o, /i, • • • , /m and use (7.7). 

The first thing we will compute is f-^Df,{s). We already have the expression (6.11) for Df, 
in terms of our local coordinates. So we need the expression for the function fg. In analogy 
to (4.13) we set 

Complex conjugation on g preserves Omin and p and so preserves Y by (3.7). This induces 
naturally a complex conjugation map / i— > / on R{Y). This is then a C-anti-linear real algebra 
involution. 

Proposition 7.4. The unique expression for fe in terms of our local coordinates fo, fi, ■ ■ ■ , fm 
on Y is 



/e = /o = ^ (7.20) 

Jo 



Proof. This is proven in the same way as in [B-K4, Prop. 4.3]. I 

To state and prove our computation of f-De{s), we need to encode the monomial A''* into 
a new vector, namely the multi-degree deg{N^). We define this by 

deg(7V,„+,.) = (0,...,0,2,...,2,0,...,0), 1 < j < Qn (7.21) 

where there are c„ zeroes follows by j twos followed by zeroes, and 

g 

deg(iV*) =^tideg(iVi) (7.22) 

i=l 

where addition of vectors is component-wise. If /x = (/xi, . . . ,/Xg) = deg(A''*) then 

Ml + h = 2 deg AT* = 2z (7.23) 

Let 5 be the g- vector such that (5c„+j = dn{qn — j) for I < j < Qn where we recall the root 
multiplicity numbers d„ from the proof of Proposition 5.7. So 

5 = (5i,..., Sa) 

^ (7.24) 
= {di{qi - l),...,di, 0,^2(^2 - l),...,d2,0, ...d^(g^- l),...,d^,0) 

Finally we define the Q'-vector 

v = {vi,...,Vq) = {wi,...,wi,...,we,...,we) (7.25) 
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f^N^VX of (7.12) and n = 



(7.26) 



(7.27) 

Proof. First (6.11) and (7.20) give 

f^D, = NP{£o„...,jC-oJ (7.28) 

where dk = -§j^ for A; = 1, . . . , m. To compute feD^ (s) we first compute Cq^. (s) . But dk (/o) = 
and also taking the Lie derivative of (5.17) we get Cd^,{\fk) = 0. So we get 

>Ca,(5) = dk{fSN')VA + fSN\£9^VA) = f^{dkN')y/A (7.29) 

Then (7.28) gives 

hD,{s) = f^N (P(ai, . . . ,9^)(iV*)) VA (7.30) 
Next we introduce the graded complex algebra isomorphism 

S{h)^C[di,...,dm], B^Ob (7.31) 

defined in degree Ihy = fQ^rj^, x G ^i, so that, by (3.19), d^^. = -§j^. Then we can rewrite 
(7.30) as 

hD,{s) = f^N {dp{N')^ (7.32) 
It follows easily, as in [B-K4, (4.4.4)], that 

Ndp{N^) = Nd^*^) (7.33) 

Indeed, it suffices to check that d^y = d^y where x e ti and y G We find d^y = 

fr^U{[x,y])e = tp{[x,y]) = {x,y)g = d^y. 

Now we can compute Ndp{N^) using the generalized Capelli Identity of Kostant and Sahi 
[K-S]. This is similar to [B-K4, proof of Th. 4.4], but we are in a more general situation here 



where each Wn occurs q„ times. 

From now on, we assume that s G is of the form s 
deg(iV*). 

Proposition 7.5. We have 

q Vi-1 
i=l i=0 

where Cij{iJ,) is the Capelli multiplier given by 
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where the Jordan algebra t-i is not necessarily simple and also the multiphcities wi, . . . ,Wi 
may be non-trivial. The point is that Ndp{N^) breaks into a product with one factor for each 
simple component of The nth factor is 

NZ\,„ 5p^- Y • • • ) (7-34) 

The Capelli Identity for j„ says that the operator N^J,^ dp-^n acts on it argument in (7.34) 

by a scalar and computes that scalar. 

Putting all the factors together we obtain 

q Vi-1 

Ndp{N') = iV* n n (7-35) 

i=l j=0 

which then gives (7.26) because of (7.32) and (7.33). 

There is one subtle point here: the appearance of the factor in Ci_j{fi). The corre- 
sponding Capelli multiplier from [K-S] is just ^(/Ltj + 6i — 2j). However the factor v^^ arises 
because of the way we have paired 6_i with ti. Let = + ■ ■ ■ + e| be the decomposition 
of corresponding to (4.11) so that is the Jordan identity element in similarly we get 
et = + • • • + e|. Then we easily find 

e 

(e«,ei)0 = 5^(e^,e^)0 and (e^, 6^)0 = g„u;„ (7.36) 

n=l 

This fits with (4.14) since (61,6^)0 = 4 just as in [B-K4, proof of Theorem 4.4.) So we get 
9e^(e^) = qnWn- Howcvcr the normalization from [K-S] is that 9ey(e^) = Qn- The ratio Wn 
then appears in the denominator of our Capelli multiplier. ■ 

Let S be the set of ordered pairs occurring in (7.26). The cardinality of S is qiW\ + 
... + g„u;„ = 4by (4.14). 

Corollary 7.6. We have 

D,k{s) = s W {CiM + l) (7.37) 



Proof. As in the last proof we find, cf. (7.28), 

D,h = P{Co„...,CajN (7.38) 

and so, as in (7.32) and (7.33), 

D,h{s) = f^idANN^))^ (7.39) 
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Then we find 

dp{NN')= n C,,,{^i') (7.40) 

(i,j)e2 

where /i' = deg(AfAr*). But /x' = 2v + and so Cij{n') = Ci,j(At) + 1- Now we get (7.37). ■ 

Next we want compute feDe{s). We solve this as in [B-K4] by introducing, in the next 
result, an involution 9 of H. We construct this using the group element 

0o = exp^{ei-et)eK (7.41) 

where exp : t ^ K is the exponential map. The same arguments used in [B-K4, Lem. 4.6, 
Prop. 4.6 and Th. 4.7] give 

Lemma 7.7. The action ofOo on p preserves Y and defines a graded complex algebra involution 
9 of R{Y) which commutes with complex conjugation. We have /q = ±/o- 

The natural action of Og on the K -homogeneous half- form bundle Na over Y defines a 
complex linear involution 9 : H ^ H . Then 9 : H ^ H preserves the grading (5.15) and is 
compatible with the R{Y) -module structure so that {fs)^ = f^s^. 

9 permutes the simple lo-submodules in H and moves lowest weight vectors to highest 
weight vectors. For any s e H we have 

f,D^{s) = ifeDeis'))' (7.42) 

We can now prove 
Proposition 7.8. We have 

feD^{s) = s H {r-l-Q,,ifi)) (7.43) 



Proof. Lemma 7.7 reduces the problem to computing fjDe{s^). Now s is a lowest weight 
vector in some simple fio-submodule F in and so Lemma 7.7 implies that is a highest 

weight vector in the simple ^o^submodule F^ in H. Then there is a lowest weight vector 
s* e F^ (unique up to scaling). We can write s* = fgu^/X where u G (S'''(t_i))"'o . Let 
//* = deg(u). Then 

f^D,is') = s' n ^^Af^*) (7-44) 

We claim that there is an involution {i,j) on the set E such that 

Ci^j^{fi*) = r-1-Ci,j{fi) (7.45) 
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This, because of Lemma 7.7, gives (7.43). 

The construction of the involution requires several calculations. To carry these out, we 
bring to the forefront the theory of weights associated to our triple (t, to,6') from §4. Indeed, 
our choice of {i),b) in §5 was compatible with complex conjugation and the complex Cartan 
decomposition to = ® i^ so that we have a complex conjugation stable splitting f) = a © t 
where a C r is a maximal abelian subalgebra and t C 

Now dime o. = q and a* has a unique basis ei, . . . ,£q such that the a-weight of Nc„+j, 
where 1 < j < Qn, is — 2(£c„+i + . . . + £c„+j)- The weights are pure imaginary and the 
action of 9o gives the complex involution 6 of to with fixed algebra t' and (— l)-eigenspace r. 
In particular, 6o acts as —1 on a. It follows that 

a = a-weight of s ^ —a = o-weight of (7.46) 

Let 1/ 1-^ u'^ he the involution of o* which exchanges the highest weight of a simple to- 
submodule in S{i-i) with the lowest weight. Then 

-a'^ = o-weight of s* (7.47) 

In terms of our basis of a* we have £c„+i — ^c„+g„-i+i for 1 < i < Qn- The o-weight of /o = fe 
is ipla, which we again call ■i/'- From now on, we identify a vector i/ = (i^i, . . . , fq) with the 
o-weight V = Yh^i ^i^i- 

We now define our involution on S by 

£i. = el and f = Vi - j - 1 (7.48) 

(This automorphism of H is can happen to be the identity, so by involution we mean just that 
the order divides 2.) Then fij = fii*. 

Let us put g = so that s = fogy/A , n = deg{g) and g G (S"=(ti))'"o" . Then the 
a-weight of g is zip — ^x. Also 

777/ I J_ 1^ 

A = o-weight of vA = — ^ — V - ^ (7.49) 

where —n is the weight of o on A'^Li. The sum of the o-weights of /q , g and a/A is 

a = {p + z)ijj -iJ, + \ = rijj-iJ,-- (7.50) 

Now we can compute The sections s = f^g^^ and s* = /qU^/A have the same 
eigenvalues under E' and C^h which means that p + z = a + b and 2p + z = 2a + b. Hence 
p = a and z = b. So the o-weight of ii is z^jJ — fi* and we get 

-a^ = a-weight of s* = {p + z)ip - n* + X (7.51) 

Applying r to (7.49) and subtracting (7.50) we get 

= 2a^ + fx^ = 2rtP - - K (7.52) 
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To obtain a proof of (7.45), we write out (7.52) in terms of components. The a-weight ijj 
has the same components as the vector v in (7.25). A key observation is that the components 
of K are Ki = 2 + 5i + SJ . To see this, we start from the fact that k is the sum of the a- weights 

of the top exterior powers of the spaces jj^j. The weights of o on are precisely the 
weights 2ei and Si + ej where c„ + 1 < i < j < c„ + g„. So = 2J2.ei + dn J2i<j{^i + ^j) 
and this gives our formula for Ki. Now (7.52) gives 

fil +Si = 2rvi - 2 - (//[ + 6J) (7.53) 

Now subtracting 2j from both sides of (7.53) and using (7.48) we get 

III +Si- 2j = 2{r - l)vi - ifii, + Si* - 2f) (7.54) 

Dividing through by 2vi (notice Vi = Vi*) we get (7.45). ■ 

Arguing as in the proof of Corollary 7.6 we get 
Corollary 7.9. We have 

Defe{s) = s n (^-a,,(/x)) (7.55) 
(»J)es 



We can now compute the scalar R from (7.3); we already computed X in (7.14). Starting 
with (7.18) and plugging in (7.26), (7.37), (7.43) and (7.55) we get 

^ _ \{C^M \{{C^M + 1) n(r - 1 - C,M) mr - C.M) .756^ 
(r-l)r r(r + l) (r - l)r r(r + 1) \ - > 

This is only valid when r ^1 (as we know r > 0). 

Fortunately, the expression for R in (7.56) simplifies greatly. We can apply the following 
formal identity given [B-K4, Lem. 4.8]. Put 

J{ai;b) = J{ao,ai,a2,a3;b) = ('-57) 

and a'^ = b — where b, ao, 01,02,03 are five indeterminates. Then 

Jim- b) - J{a'i- b) - J{ai + 1; 6 + 1) + J(a^ + 1; 6 + 1) 
= 2b— (ao + ai + 02 + 03) 

Applying this with b = r — 1 and ai, 02, 03, 04 set equal to the four Capelli multipliers Cij{fj,) 
(taken in any order), we get 

R = 2r-2- J2 (7-59) 
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To prove (6.22), we need to show R = X. We compute 



7^1 J^o .=1 ^ (7.60) 

m 

=z + — -2 = 2r-2-X 

The second to last equahty follows as fii = 2z, X^^^^ ~ Yln=i dn{Qn — ^)Qn/'^ = m — q 

and ^^^^ Vi = Yli=i QnWn = 4, while the last follows by (7.14). SoR = X. Thus (6.22) holds 
on s if s <^ Hi. This proves Theorem 6.3 in all cases where r = 1 never occurs in the spectrum 
of E' on H. 

Now suppose that r = 1 does occur and s G Hi. Then Proposition 5.5 implies that tq = 1 
so that Hi is the vacuum space. Thus by £^'-degree D^s = D^s = 0. Then (7.26) implies that 
YlCij{iJ,) = and so Cij{iJ,) = for some But also (7.56) collapses to 

^ = ^ 11(1 - CM) - i Hic.M + 1) 

But (7.60) gives J2 ^ijil^) = Hence i? = X if and only if the third elementary symmetric 
function of the four numbers Cij{n) is zero. But we already know at least one Cij{iJ,) vanishes. 
So i? = X if and only if at least two of the four numbers Ci^j{fi) are zero. 

At this point we observe that r = vq = 1 implies something very particular about the form 
of s: in our normal form s = /q7V*\/A we have 

N' = iV-^ iV--f • • • iV--:"^,, (7.62) 

where < u„ < Now it follows from (7.27) that the list of four numbers Cij{iJ,) has at 
least £ zeroes. Thus we are left with the case where £ = I. 

Suppose £ = 1. Then m < 4. To see this, we consider a highest weight vector si G Hi for 
the 6-action. Then si = /o"''-\/A for some j G The eigenvalue of C^h on si is — 2j+(m+2)/2 
and must be non-negative. The eigenvalue of E' on si is — j ' + (m + l)/2 and is equal to 1. 
But then (m + 2)/4 > j = (m + l)/2 - 1 and so 4 > m. 

Now looking at Table 4.6, we see that £ = 1 and m < 4 only if = 5[(p, M) where 
p = 6,5, or 3. We rule out p = 5 because of Proposition 5.7(ii). For p = 6 we have rg = 1 
when Sq = Aq and Hi ~ C. Then s = sq and n = {111,112) = (0,0). The multipliers Cij{iJ,) 
are and this list has two zeroes as required. For p = 3, we have 

ro = 1 when Sq = /qAq and Hi c± S^C'^. Then s is one of four vectors with n = (ni) where 
/ii = 0,2,4, or 6. The multipliers Cij{p) are ^, and so we never get two 

zeroes in this list. Thus this one case fails to produce a representation. This concludes the 
proof of Theorem 6.3. 

Next we finish the proof of Theorem 6.8. We started this in §6, and left off at (6.32) 
where we needed to compute the numbers 7fe defined by Te(/o sq) = 7fe/o~^so. But now we 
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can compute the 7^ because of Proposition 7.8. Indeed, let s = /qSq; then r = Vq + k and 
/X = deg(s) = 0. Now (7.4.3) gives 

T.(/„U) = /o--»o "'-'7;^,_^(^^^,;^" (7.63) 

We can simphfy the factor 7^ appearing in (7.63) by computing the four numbers Cij{0). 
We have Te(so) = T^^so) = since sq is a vacuum vector. Hence, if ro 7^ 1 then the list of 
four multipliers Cij{0) contains zero and ro — 1. Moreover we just showed that if ro = 1 then 
the list contains zero with multiplicity at least two. Thus, regardless of the value of ro, we can 
write the list of four multipliers Cij{0) as 0, ro — 1, ro — a, ro — 6 where a and b are unknown. 
Then (7.63) gives 

Consequently 

71 • • • 7n = I ... 7-65 

(ro + l)n 

Because of (6.32) this gives (6.30). In fact we have gotten the more precise information 
Proposition 7.10. We have equalities of multi-sets: 

{Ci j(0)}(i,,)eH = I I = {0, ro - 1, ro - a, ro - b} (7.66) 

In this way P and vq determine uniquely the numbers a and b appearing in Theorem 6.8. 

Notice that the choice of 0r determines P while the choice of determines ro. 

Proposition 7.10 implies Ci,j (0) = 3ro - 1 - a - 6. But also ^ Cij(O) = 2ro - 2 - 
by (7.60). Comparing, we get (6.31). 

Finally we can use Proposition 7.10 to compute the numbers a and b in Table 6.9. We 
then observe that a and b are always positive. But we also have a nice theoretical proof of the 
positivity. 

By (7.66), the four numbers ro — (^^^(O) are l,ro,a, 6. Using first (7.45) and then (7.27) 
we can write 

ro - C\j{0) = 1 + J. (a) = — (7.67) 

where a = 0*. The last expression in (7.67) is positive since ai*,Si* > and Vi* > j*. This 
concludes the proof of Theorem 6.8. 



8. The Reproducing Kernel of H. 
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The aim of this section is to show that the Hilbert spaces 7i carrying the unitary irreducible 
representations constructed in §6 each admit a reproducing kernel IC and is a holomorphic 
half-form on y x y. It follows then that H, consists entirely of holomorphic half-forms on Y. 

We work in the setting of Theorems 6.3, 6.6 and 6.8. (So the one case Gk = S'L(3,M), 
ro = 1, Hi ^ is excluded as this case did not quantize.) 

To begin with, we explain how the notion of reproducing kernel applies here. Our Hilbert 
space H. is the completion of H = r{Y, N2 ). Therefore, using the grading (5.15) of H, we may 
regard a section s G as a formal sum 

s= J2 (8.1) 

n€Z+ 

where s„ G i?ro+n- Then s is a holomorphic section of if and only if the series in (8.1) 
converges locally uniformly. 

1 

The complex conjugate space H identifies naturally with the space r(y, ) of algebraic 
i_ 1^ 

holomorphic sections of over Y. Here is the complex conjugate line over the complex 
conjugate algebraic manifold Y. So we get an identification 

H^H = r{Y xY,N^ ^W) (8.2) 

A reproducing kernel for H is a section K. of (g) over Y x Y such that for each 
V G y, the formula 



/C^(ii) = IC{u,v) (8.3) 

1 

defines a section fC^ eH^ N| and we have the "reproducing" property for all s eH 

siv) = {s\JC,) (8.4) 
1^ 

This makes sense as both sides of (8.4) define vectors in N^^ . 

H admits a reproducing kernel if and only if the evaluation map s s{v) is continuous 
on H for every point v eY. A reproducing kernel on H, if it exists, is unique and is computed 

by 

IC = ^gk0gk (8.5) 
k 

where {gk} is any orthonormal basis of H. See , e.g., [F-K, IX, §2] for the case of Hilbert spaces 
of holomorphic functions. 

Each space Hr„+n, n G Z+, is finite-dimensional and so admits a reproducing kernel 
n„ G r(y x y, (g)N^ ). The reproducing kernel KoiH exists if and only if /C = Ylnez+ -^n, 
i.e., if and only if the series X^„gz n„ converges. 



52 



We have a i^-invariant function T G R{Y x Y) defined by 

T{u,v) = {u,v)q (8.6) 
Then u ^ T{u,u) is a positive real function on Y. 

Theorem 8.1. For any orthonormal basis {gk} of H, the series in (8.5) converges locally 
uniformly and moreover we have the formula 

IC= iF2(ro + l;a,6;r)no (8.7) 

where a and b are as in Theorem 6.8 and Hq is the reproducing kernel of Hrf^. Consequently, 
K is a holomorphic section 

/CGr''°'(y xF,n4 ®N^) (8.8) 



Proof. For each n, n„ is a i^-invariant section of (8) N^. This follows as the Hermitian 
inner product on if^o+n is -fiTiR-invariant. Hence the quotient Ilri/no is a ivT-invariant rational 
function on Y x Y. We have natural actions of i^T x C* on y and Y where C* acts by the 
Euler scaling action. 

Lemma 8.2. The product action of K x C* on the variety Y xY has a unique Zariski dense 
orbit W. The function T separates the K -orbits in W. 

Moreover any K -invariant rational function onY xY is a polynomial in T and T~^. 

Proof. The isotropy group of K at (e,e) is n = = K'q. So the /^-orbit of 
(e,e) is isomorphic to K/Kq and hence has codimension 1 in y x y by Theorem 4.1 since 
dimcy X y = 2 dime y = dimcO. The C*-orbit of (e,e) and the if-orbit of (e,e) meet in 
exactly two points, namely ib(e,e). This follows from the easy fact that (a • e,a • e) = {te,te) 
if and only if a G and x(^) = i = x{^)~^- 

In particular then the C*-orbit and the iC-orbit are transverse at (e,e). So by dimension, 
the orbit W of (e,e) under iiT x C* is Zariski dense in y x y. 

Now the (punctured) line {{te,te) \ t E C*} meets all the J^-orbits in W and the function 
separates out the points lying in different ivT-orbits. But the function T is i^-invariant 
and satisfies T{te,te) = t^ . So T separates the iC-orbits and the last assertion of the Lemma 
follows easily. ■ 

Lemma 8.2 implies that n^/IIo is a polynomial in T and T~^. But also n^/IIo is bi- 
homogeneous of degree (n, n) under the scaling action of C* x C* on y x Y. Since T is 
bihomogeneous of degree (1, 1), it follows by bihomogeneity that 

n„=p„r"no (8.9) 

for some scalar Pn^C*. 
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Our problem now is to compute the scalars Pn- We will do this by writing out the "leading 
terms" of n„ , T" and Ho . We formulate a notion of leading term in the following way. Suppose 
F is a highest weight representation of K of weight k and S & V 0V is iiT-invariant. Then we 
can write 5 as a sum of weight vectors Sa where the weight of each Sa is of the form (a, —a). 
Then we call the term of weight (k, —k) the leading term. 

We can identify R{Y xY)= R{Y) (g) R{Y) and then we have the expansion 

r 

T = Y.f-^^77, (8.10) 

k=l 

where ui, . . . ,Ur is any basis of p which is orthonormal with respect to the Hermitian inner 
product on p given by {ui\uj) = {ui,Uj)g. Choosing ui, . . . , u^. to be an orthonormal basis by 
weight vectors, we find (recall /e = /o and (e,e)g = 1) 

leading term oi T = fo <^Jo (8-11) 



and also 

leading term of T" = (g) (8.12) 



Next we choose an orthonormal basis on i^^o+n consisting of weight vectors. This basis 
then contains the highest weight vector /o^so/||/o''So|| and we find 



leading term of n„ = ^ -^f ° (8.13) 

ll/o"so|| 

The leading term of a product is the product of the leading terms, and so equating leading 
terms in (8.9) we get 



ll/o«o||' 



since ||so|| = 1- So using (6.30) we find 



Pv 



1 ^ (ro + l)n 
||/o"so||^ n!(a)„(6)„ 



^.15) 



Thus 



En. 

nez+ 



E 



(rp + 1)„ 
n\{a)n{b), 



-r"no= iF2(ro + l;a,6;r)no 



(8.16) 



This proves (8.7). The hypergeometric series here has infinite radius of convergence, and so 
i-f2('^o + 1; b; T) defines a holomorphic function on 1" X Y". Thus /C is a holomorphic section 
over Y X Y. This concludes the proof of Theorem 8.1. ■ 
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Theorem 8.1 easily gives 



Corollary 8.3. Ti consists entirely of holomorphic sections 0/N2 and K. is the reproducing 
kernel ofH. 



59. Examples of the Quantization. 



A feature of our results is that we can construct the representation tt in any model of H 

so long as we are given both the i^-module structure and the R{Y)-modu\e structure on H. 
In particular the half-forms can be completely suppressed in the model. We illustrate this by 
2 examples. These cases are particularly simple ones where the polynomial P factors in (4.13) 
into a product of 4 linear terms. 

Example 9.1 Let 0k = so(4, 4); this is Case (ix) in Tabic 6.9 with p = q = 4. Then 
t = s((2,C)®'^. As K-modules we have H c:^ R{Y) ~ ©„>o5'"(C2)®4. a model of H is given 
in the following way. Let S be the polynomial ring in 8 variables Xp^i where p G {1, . . . , 4} and 
i G {1, 2}. Then H is the subalgebra of S generated by the 16 products xi^iX2,jX^^kXA,i where 
i,j,k,l G {1,2} so that 

H = ©n>oC„[xi,i, 331,2] • 'Cn[x2,l,X2,2] ' C„ [0:3,1 , 0:3,2] • C„ [^4,1 , X4,2] C S 

where C„[n, v\ is the space of degree n polynomials in u and v. Notice then that H is the space 
of invariants in S under a scaling action of C* x C* x C*. Let /? be the differential operator 
on S given by 

P = Xl^l- h Xi,2^ h 1 

dXi,! dXi,2 

Then the following 28 pseudo-differential operators on S preserve H and satisfy the bracket 
relations of so (8, C). I.e., these 28 operators form a basis of a complex Lie algebra q isomorphic 
to so(8,C). 



d d d d , r <^ r, ,T 

Xp,i^ , Xp^2^ , Xp^i- Xp^2^ where p G {1,2,3,4} 

dxp^2 dxp,i dxp^i dxp^2 

Xl,iX2,jX3^kX4,l - 



(5{f3 + 1) dxi^i,dx2^j'dx-i^k'dxi^i/ 
where {i,i'} = {j,/} = {k,k'} = {1,1'} = {1,2} 
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Then ro = a = 6 = 1 in Table 6.9 and so the flK-invariant inner product on H satisfies 

(l)n(l)n ^ 1 

n!(2)„ (n + 1) 
where p G {1, . . . , 4} and i G {1,2}. This agrees with the result in [K]. 

Example 9.2. Let Qm. be of type G2; this is Case (viii) in Table 6.9. Let S be the 
polynomial ring in 4 variables ui,U2,xi,X2 and let S' ~ R{y) be the subalgebra generated by 
the 8 products u^Xj and ufui'Xj, where = {1,2} and j G {1,2}. A model of H is the 

S"-submodule 

H = en>oC3n+2['"i,^^2] " C„[a;i,a;2] C S 
Let [3 be the differential operator on S given by 

P = Xi-^ h h 1 

0X2 0X2 

The following 14 pseudo-differential operators on S preserve H and satisfy the bracket 
relations of G2 so that they form a basis of a complex simple Lie algebra of type G2. 



nl 



d 



d 



d 



■"1^ ' "2-^ , Ui- U2-^ 

OU2 OUi OUi OU2 



^2^, Xi- 2:2;^— 

0x2 oxi 0x1 0x2 



27P{(3+l)dul,dxf 



27/?(/3+ 1) dul,duidxj 



where {z,z'} = {j, j'} = {1,2} 



where {i,i'} = {j,/} = {1,2} 



Then ro = 1, a = 4/3 and 6 = 5/3 in Table 6.9 so that the 0R-invariant inner product on 
H satisfies 



n! 



(4/3)„,(5/3), 



(3/) +3)! 



n!(2)„ 33»3!n!(n + l)!(n+l)! 



where i,j G {1, 2}. 
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